[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:06 dist-test-slave-42z9 krb5kdc[10973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:06 dist-test-slave-42z9 krb5kdc[10973](info): set up 2 sockets
Aug 09 13:09:06 dist-test-slave-42z9 krb5kdc[10973](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 13:09:08.069247 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:08 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:08.246120 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:42089
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:42089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:08.573009 10992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:08.573297 10992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:08.573426 10992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:08.580523 10992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:08.580626 10992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:08.580698 10992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:08.580767 10992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:08.580830 10992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:08.590441 10992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:42089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:42089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:08.591722 10992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:08.593043 10992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:08.604717 11003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:08.604753 11000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:08.604662 11001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:08.605057 10992 server_base.cc:1034] running on GCE node
I20260809 13:09:08.606145 10992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:08.607514 10992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:08.608717 10992 hybrid_clock.cc:648] HybridClock initialized: now 1786280948608667 us; error 99 us; skew 500 ppm
I20260809 13:09:08.609055 10992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:08 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280948, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:08.612099 10992 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:08.613549 10992 webserver.cc:460] Webserver started at http://127.10.181.190:33803/ using document root <none> and password file <none>
I20260809 13:09:08.614240 10992 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:08.614362 10992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:08.614660 10992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:08.617856 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "81f5343baf364132a3a327f62b645c94"
format_stamp: "Formatted at 2026-08-09 13:09:08 on dist-test-slave-42z9"
server_key: "2c4e04776201afd1902bcd0de14e3c69"
server_key_iv: "3bd7bd8210ec6b271971b2bee50ae055"
server_key_version: "encryptionkey@0"
I20260809 13:09:08.618605 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "81f5343baf364132a3a327f62b645c94"
format_stamp: "Formatted at 2026-08-09 13:09:08 on dist-test-slave-42z9"
server_key: "2c4e04776201afd1902bcd0de14e3c69"
server_key_iv: "3bd7bd8210ec6b271971b2bee50ae055"
server_key_version: "encryptionkey@0"
I20260809 13:09:08.623807 10992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:08.627341 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:08.628604 10992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:09:08.628790 10992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "81f5343baf364132a3a327f62b645c94"
format_stamp: "Formatted at 2026-08-09 13:09:08 on dist-test-slave-42z9"
server_key: "2c4e04776201afd1902bcd0de14e3c69"
server_key_iv: "3bd7bd8210ec6b271971b2bee50ae055"
server_key_version: "encryptionkey@0"
I20260809 13:09:08.628978 10992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:08.650231 10992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:08.654361 10992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:08.654611 10992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:08.672200 10992 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:42089
I20260809 13:09:08.672223 11085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:42089 every 8 connection(s)
I20260809 13:09:08.673538 10992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:08.677219 11086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:08.681550 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10992
I20260809 13:09:08.681880 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:08.684396 10966 external_mini_cluster.cc:1428] Setting key 06642e5d482b85fbba01e727cb641643
I20260809 13:09:08.690366 11086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94: Bootstrap starting.
I20260809 13:09:08.694392 11086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:08.695716 11086 log.cc:824] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:08.698952 11086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94: No bootstrap required, opened a new log
I20260809 13:09:08.710115 11086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } }
I20260809 13:09:08.710549 11086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:08.710667 11086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f5343baf364132a3a327f62b645c94, State: Initialized, Role: FOLLOWER
I20260809 13:09:08.711321 11086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } }
I20260809 13:09:08.711620 11086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:08.711818 11086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:08.712015 11086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:08.714910 11086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } }
I20260809 13:09:08.715526 11086 leader_election.cc:302] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81f5343baf364132a3a327f62b645c94; no voters:
I20260809 13:09:08.716324 11086 leader_election.cc:288] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:08.716394 11091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:08.717667 11091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [term 1 LEADER]: Becoming Leader. State: Replica: 81f5343baf364132a3a327f62b645c94, State: Running, Role: LEADER
I20260809 13:09:08.718187 11091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } }
I20260809 13:09:08.718696 11086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 13:09:08 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:08.720775 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f5343baf364132a3a327f62b645c94. Latest consensus state: current_term: 1 leader_uuid: "81f5343baf364132a3a327f62b645c94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } } }
I20260809 13:09:08.721163 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:08.725329 11092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81f5343baf364132a3a327f62b645c94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f5343baf364132a3a327f62b645c94" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42089 } } }
I20260809 13:09:08.725700 11092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:08.725780 11103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:08.732103 11103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:08.733743 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:08.689323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.10.181.190:42089)
0809 13:09:08.690275 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:08.690293 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:08.691554 (+ 1261us) server_negotiation.cc:407] Connection header received
0809 13:09:08.704871 (+ 13317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:08.704895 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:08.705322 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:08.705704 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:08.709498 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:08.711015 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:08.713393 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:08.714115 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:08.714253 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:08.722011 (+ 7758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:08.722025 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:08.722047 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:08.722104 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:08.726425 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:08.727443 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:08.727453 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:08.727461 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:08.727665 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:08.729310 (+ 1645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:08.729315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:08.729318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:08.730093 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:08.730385 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:08.731656 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0809 13:09:08.732203 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":396,"threads_started":1}
I20260809 13:09:08.743940 11103 catalog_manager.cc:1347] Generated new cluster ID: d6ce92eb2ddd40fda4e4f27a2e5b1141
I20260809 13:09:08.744046 11103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:08.751358 11103 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:08.753096 11103 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:08.763904 11103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81f5343baf364132a3a327f62b645c94: Generated new TSK 0
I20260809 13:09:08.764853 11103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:08.812956 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:09.146631 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:09.146910 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:09.147059 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:09.156064 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:09.156186 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:09.156350 11134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:09.165915 11134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:09.167160 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:09.168694 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:09.181576 11141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:09.181612 11143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:09.181741 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:09.182715 11134 server_base.cc:1034] running on GCE node
I20260809 13:09:09.183306 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:09.184214 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:09.185417 11134 hybrid_clock.cc:648] HybridClock initialized: now 1786280949185371 us; error 33 us; skew 500 ppm
I20260809 13:09:09.185760 11134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:09 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:09.188807 11134 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:09.190267 11134 webserver.cc:460] Webserver started at http://127.10.181.129:34965/ using document root <none> and password file <none>
I20260809 13:09:09.190907 11134 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:09.191017 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:09.191294 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:09.194293 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "aa4055d1cdc44383bec7d2336c769c44"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "651dffdeb3b1662bb90e93f190baea0a"
server_key_iv: "9582076b49e54cdc042884dedb48047f"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.195013 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "aa4055d1cdc44383bec7d2336c769c44"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "651dffdeb3b1662bb90e93f190baea0a"
server_key_iv: "9582076b49e54cdc042884dedb48047f"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.200073 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:09:09.203501 11157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.204741 11134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:09:09.204903 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "aa4055d1cdc44383bec7d2336c769c44"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "651dffdeb3b1662bb90e93f190baea0a"
server_key_iv: "9582076b49e54cdc042884dedb48047f"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.205082 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:09.221465 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:09.225224 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:09.225517 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:09.226450 11134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:09.227840 11134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:09.227929 11134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.228040 11134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:09.228085 11134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.258576 11134 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:34617
I20260809 13:09:09.258631 11325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:34617 every 8 connection(s)
I20260809 13:09:09.259865 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:09.260911 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11134
I20260809 13:09:09.261147 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:09.261956 10966 external_mini_cluster.cc:1428] Setting key 4f37d5f4999b4c019324b9dbba90c020
Aug 09 13:09:09 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:09.282522 11328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:09.262761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:49835 (local address 127.10.181.190:42089)
0809 13:09:09.263435 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:09.263439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:09.264437 (+ 998us) server_negotiation.cc:407] Connection header received
0809 13:09:09.266946 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:09.266951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:09.267013 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:09.267108 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:09.269080 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.269857 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.271775 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.272221 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.272336 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:09.276055 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:09.276062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:09.276066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:09.276111 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:09.279299 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:09.280081 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:09.280088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:09.280092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:09.280250 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:09.280636 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:09.280641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:09.280644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:09.281016 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:09.281193 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:09.281886 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 13:09:09.282142 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":476,"threads_started":1}
I20260809 13:09:09.283244 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:09.263357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42089 (local address 127.10.181.129:49835)
0809 13:09:09.264256 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:09.264309 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:09.266665 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:09.267370 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:09.267382 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:09.268097 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:09.268898 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:09.268924 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.270020 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.270025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:09.271633 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:09.271647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.272454 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.272463 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:09.272684 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:09.273301 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:09.273335 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:09.275835 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:09.279450 (+ 3615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:09.279461 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:09.279479 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:09.279956 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:09.280350 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:09.280353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:09.280356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:09.280535 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:09.281137 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:09.281143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:09.281495 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 13:09:09.281751 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 13:09:09.282155 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":327,"threads_started":1}
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
I20260809 13:09:09.284776 11326 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42089
I20260809 13:09:09.285087 11326 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
I20260809 13:09:09.285866 11326 heartbeater.cc:511] Master 127.10.181.190:42089 requested a full tablet report, sending...
I20260809 13:09:09.288267 11042 ts_manager.cc:194] Registered new tserver with Master: aa4055d1cdc44383bec7d2336c769c44 (127.10.181.129:34617)
I20260809 13:09:09.290171 11042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:49835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:09.307945 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:09.635658 11334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:09.635931 11334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:09.636071 11334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:09.643265 11334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:09.643380 11334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:09.643535 11334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:09.652879 11334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:09.654147 11334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:09.655455 11334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:09.668123 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:09.668174 11343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:09.668342 11339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:09.669189 11334 server_base.cc:1034] running on GCE node
I20260809 13:09:09.669857 11334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:09.670773 11334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:09.671965 11334 hybrid_clock.cc:648] HybridClock initialized: now 1786280949671920 us; error 37 us; skew 500 ppm
I20260809 13:09:09.672286 11334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:09 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:09.675489 11334 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:09.677089 11334 webserver.cc:460] Webserver started at http://127.10.181.130:46567/ using document root <none> and password file <none>
I20260809 13:09:09.677812 11334 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:09.677932 11334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:09.678244 11334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:09.681232 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "c8b3fab86ca942a9a9b72d8e6dd83de7"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "d616f66c333a14af80f98feb1e333d08"
server_key_iv: "100bd0c27b77418738d6b3456caab529"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.682058 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "c8b3fab86ca942a9a9b72d8e6dd83de7"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "d616f66c333a14af80f98feb1e333d08"
server_key_iv: "100bd0c27b77418738d6b3456caab529"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.687327 11334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:09.690836 11353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.692178 11334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:09.692328 11334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "c8b3fab86ca942a9a9b72d8e6dd83de7"
format_stamp: "Formatted at 2026-08-09 13:09:09 on dist-test-slave-42z9"
server_key: "d616f66c333a14af80f98feb1e333d08"
server_key_iv: "100bd0c27b77418738d6b3456caab529"
server_key_version: "encryptionkey@0"
I20260809 13:09:09.692512 11334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:09.706815 11334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:09.710547 11334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:09.710806 11334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:09.711717 11334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:09.712976 11334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:09.713055 11334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.713183 11334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:09.713235 11334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:09.744257 11334 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33203
I20260809 13:09:09.744302 11527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33203 every 8 connection(s)
I20260809 13:09:09.745554 11334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:09.753158 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11334
I20260809 13:09:09.753340 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:09.753870 10966 external_mini_cluster.cc:1428] Setting key fc3cdc4619103e85aad3a5c134191722
Aug 09 13:09:09 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:09.765239 11328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:09.747994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:48869 (local address 127.10.181.190:42089)
0809 13:09:09.748222 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:09.748228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:09.749393 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 13:09:09.751174 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:09.751179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:09.751250 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:09.751336 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:09.752709 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.753514 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.755520 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.756013 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.756138 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:09.759804 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:09.759810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:09.759813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:09.759845 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:09.762229 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:09.762998 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:09.763002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:09.763005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:09.763151 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:09.763559 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:09.763563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:09.763565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:09.763846 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:09.763984 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:09.764655 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 13:09:09.764885 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 13:09:09.766172 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:09.748521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42089 (local address 127.10.181.130:48869)
0809 13:09:09.749241 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:09.749278 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:09.750934 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:09.751524 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:09.751532 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:09.752015 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:09.752581 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:09.752599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.753692 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.753698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:09.755372 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:09.755388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:09.756233 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:09.756242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:09.756462 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:09.757070 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:09.757095 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:09.759581 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:09.762360 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:09.762369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:09.762385 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:09.762879 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:09.763260 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:09.763263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:09.763266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:09.763449 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:09.763973 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:09.763981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:09.764290 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 13:09:09.764537 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 13:09:09.765039 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":268,"threads_started":1}
I20260809 13:09:09.767580 11528 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42089
I20260809 13:09:09.767864 11528 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:09.768571 11528 heartbeater.cc:511] Master 127.10.181.190:42089 requested a full tablet report, sending...
I20260809 13:09:09.770160 11042 ts_manager.cc:194] Registered new tserver with Master: c8b3fab86ca942a9a9b72d8e6dd83de7 (127.10.181.130:33203)
I20260809 13:09:09.770939 11042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:48869
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:09.795878 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:10.135110 11535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:10.135370 11535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:10.135507 11535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:10.142596 11535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:10.142694 11535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:10.142848 11535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:10.152232 11535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42089
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:10.153422 11535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:10.154752 11535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:10.167280 11543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:10.167210 11542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:10.167238 11547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:10.168357 11535 server_base.cc:1034] running on GCE node
I20260809 13:09:10.168923 11535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:10.169857 11535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:10.171060 11535 hybrid_clock.cc:648] HybridClock initialized: now 1786280950170993 us; error 52 us; skew 500 ppm
I20260809 13:09:10.171360 11535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:10.174304 11535 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:10.175649 11535 webserver.cc:460] Webserver started at http://127.10.181.131:34233/ using document root <none> and password file <none>
I20260809 13:09:10.176287 11535 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:10.176395 11535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:10.176708 11535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:10.179563 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "4f823927f00744f08aea241b54465950"
format_stamp: "Formatted at 2026-08-09 13:09:10 on dist-test-slave-42z9"
server_key: "51a97c28735fbea4ae5b94ab618cc1db"
server_key_iv: "afb1272b9cc2116351485ed3fd322791"
server_key_version: "encryptionkey@0"
I20260809 13:09:10.180291 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "4f823927f00744f08aea241b54465950"
format_stamp: "Formatted at 2026-08-09 13:09:10 on dist-test-slave-42z9"
server_key: "51a97c28735fbea4ae5b94ab618cc1db"
server_key_iv: "afb1272b9cc2116351485ed3fd322791"
server_key_version: "encryptionkey@0"
I20260809 13:09:10.185168 11535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 13:09:10.188476 11558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:10.189741 11535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:10.189893 11535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "4f823927f00744f08aea241b54465950"
format_stamp: "Formatted at 2026-08-09 13:09:10 on dist-test-slave-42z9"
server_key: "51a97c28735fbea4ae5b94ab618cc1db"
server_key_iv: "afb1272b9cc2116351485ed3fd322791"
server_key_version: "encryptionkey@0"
I20260809 13:09:10.190066 11535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:10.212999 11535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:10.216743 11535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:10.217005 11535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:10.217995 11535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:10.219326 11535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:10.219421 11535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:10.219544 11535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:10.219597 11535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:10.251875 11535 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:36507
I20260809 13:09:10.251929 11741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:36507 every 8 connection(s)
I20260809 13:09:10.253149 11535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:10.253844 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11535
I20260809 13:09:10.254050 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:10.254613 10966 external_mini_cluster.cc:1428] Setting key 7b8356025975948e8471be814ba6ebf1
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:10.274876 11328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.256319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60265 (local address 127.10.181.190:42089)
0809 13:09:10.256635 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.256642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.258101 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 13:09:10.260811 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.260815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.260899 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.260980 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.262931 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.263709 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.265387 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.265835 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.265929 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.269190 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.269197 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.269202 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.269243 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.272306 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.272950 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.272954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.272956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.273068 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.273405 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.273408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.273410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.273700 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.273812 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.274477 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.274633 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:09:10.275782 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.256976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42089 (local address 127.10.181.131:60265)
0809 13:09:10.257928 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:10.257995 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:10.260489 (+ 2494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:10.261252 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:10.261263 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:10.262051 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:10.262771 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:10.262792 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.263835 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.263841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:10.265277 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:10.265289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.266018 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.266023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:10.266214 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.266856 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:10.266878 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:10.268966 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:10.272428 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:10.272435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:10.272447 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:10.272858 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:10.273149 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:10.273152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:10.273154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:10.273329 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:10.273800 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:10.273806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:10.274124 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 13:09:10.274406 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 13:09:10.274770 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":330,"threads_started":1}
I20260809 13:09:10.277100 11742 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42089
I20260809 13:09:10.277388 11742 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:10.278172 11742 heartbeater.cc:511] Master 127.10.181.190:42089 requested a full tablet report, sending...
I20260809 13:09:10.279655 11042 ts_manager.cc:194] Registered new tserver with Master: 4f823927f00744f08aea241b54465950 (127.10.181.131:36507)
I20260809 13:09:10.280431 11042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:60265
I20260809 13:09:10.283419 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:10.293314 11326 heartbeater.cc:503] Master 127.10.181.190:42089 was elected leader, sending a full tablet report...
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:10.339960 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.325108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.10.181.129:34617)
0809 13:09:10.325921 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.325930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.326037 (+ 107us) server_negotiation.cc:407] Connection header received
0809 13:09:10.326208 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.326215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.326452 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.326601 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.328021 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.329022 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.330262 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.330825 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.330939 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.334743 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.334756 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.334771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.334806 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.337632 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.338192 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.338200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.338202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.338319 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.338627 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.338630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.338632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.338956 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.339197 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.339408 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.339610 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":414,"threads_started":1}
W20260809 13:09:10.341264 11197 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35098
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:10.387634 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.374628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.10.181.129:34617)
0809 13:09:10.374858 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.374865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.375033 (+ 168us) server_negotiation.cc:407] Connection header received
0809 13:09:10.375224 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.375229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.375297 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.375385 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.376753 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.377501 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.378786 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.379256 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.379407 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.383285 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.383291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.383294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.383329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.385704 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.386254 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.386258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.386260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.386376 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.386636 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.386639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.386641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.386875 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.387052 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.387238 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.387401 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:09:10.389104 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11134
I20260809 13:09:10.400511 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.390095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35122 (local address 127.10.181.129:34617)
0809 13:09:10.390245 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.390249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.390504 (+ 255us) server_negotiation.cc:407] Connection header received
0809 13:09:10.390627 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.390631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.390684 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.390755 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.391809 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.392480 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.393700 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.394096 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.394199 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.396174 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.396179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.396182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.396211 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.398623 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.399142 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.399145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.399148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.399271 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.399533 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.399536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.399538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.399755 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.399926 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.400113 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.400254 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:09:10.401189 11318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:10.483714 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11134
I20260809 13:09:10.508102 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11334
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:10.523345 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.508991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.10.181.130:33203)
0809 13:09:10.509553 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.509560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.509596 (+ 36us) server_negotiation.cc:407] Connection header received
0809 13:09:10.509661 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.509668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.509884 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.510014 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.511113 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.512108 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.513370 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.514034 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.514137 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.518095 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.518107 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.518122 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.518155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.520883 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.521606 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.521613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.521615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.521739 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.522065 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.522068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.522071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.522394 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.522636 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.522846 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.523047 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260809 13:09:10.524708 11517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:10.606215 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11334
I20260809 13:09:10.631843 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11535
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[10973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:10.647395 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:10.632838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41846 (local address 127.10.181.131:36507)
0809 13:09:10.633527 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:10.633534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:10.633565 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:09:10.633640 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:10.633647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:10.633860 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:10.633986 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:10.635218 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.636391 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.637843 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:10.638411 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:10.638507 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:10.642362 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:10.642373 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:10.642388 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:10.642426 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:10.645089 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.645688 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.645695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.645697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.645821 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:10.646094 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:10.646097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:10.646098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:10.646422 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:10.646658 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:10.646839 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 13:09:10.647048 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":305,"threads_started":1}
I20260809 13:09:10.648734 11734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:10.729247 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11535
I20260809 13:09:10.749877 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10992
I20260809 13:09:10.751073 11079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:10.853255 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10992
2026-08-09T13:09:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3997 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[11770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[11770](info): set up 2 sockets
Aug 09 13:09:10 dist-test-slave-42z9 krb5kdc[11770](info): commencing operation
krb5kdc: starting...
W20260809 13:09:12.044348 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:12 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:12.226207 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43363
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:43363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:12.545681 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:12.545962 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:12.546100 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:12.553117 11790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:12.553216 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:12.553289 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:12.553368 11790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:12.553426 11790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:12.563202 11790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:43363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:12.564498 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:12.565862 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:12.577185 11807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:12.577261 11801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:12.577214 11803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:12.577893 11790 server_base.cc:1034] running on GCE node
I20260809 13:09:12.578615 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:12.579944 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:12.581152 11790 hybrid_clock.cc:648] HybridClock initialized: now 1786280952581062 us; error 76 us; skew 500 ppm
I20260809 13:09:12.581487 11790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:12 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280952, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:12.588477 11790 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:12.589931 11790 webserver.cc:460] Webserver started at http://127.10.181.190:42227/ using document root <none> and password file <none>
I20260809 13:09:12.590596 11790 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:12.590705 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:12.591017 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:12.594005 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "0d2777ba45e645d78ce2f28d2e1576a7"
format_stamp: "Formatted at 2026-08-09 13:09:12 on dist-test-slave-42z9"
server_key: "97aa72e5e940de51c01e38c94d97e4e5"
server_key_iv: "19e14534080e1df5345a023d10ece147"
server_key_version: "encryptionkey@0"
I20260809 13:09:12.594733 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "0d2777ba45e645d78ce2f28d2e1576a7"
format_stamp: "Formatted at 2026-08-09 13:09:12 on dist-test-slave-42z9"
server_key: "97aa72e5e940de51c01e38c94d97e4e5"
server_key_iv: "19e14534080e1df5345a023d10ece147"
server_key_version: "encryptionkey@0"
I20260809 13:09:12.599668 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:12.602980 11814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:12.604185 11790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:09:12.604342 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "0d2777ba45e645d78ce2f28d2e1576a7"
format_stamp: "Formatted at 2026-08-09 13:09:12 on dist-test-slave-42z9"
server_key: "97aa72e5e940de51c01e38c94d97e4e5"
server_key_iv: "19e14534080e1df5345a023d10ece147"
server_key_version: "encryptionkey@0"
I20260809 13:09:12.604507 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:12.622215 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:12.626339 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:12.626619 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:12.643886 11790 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:43363
I20260809 13:09:12.643895 11887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:43363 every 8 connection(s)
I20260809 13:09:12.645282 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:12.649055 11888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:12.651432 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11790
I20260809 13:09:12.651638 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:12.652277 10966 external_mini_cluster.cc:1428] Setting key bd8058cfc36af47bea3412e367bdcecf
I20260809 13:09:12.660616 11888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7: Bootstrap starting.
I20260809 13:09:12.664417 11888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:12.665712 11888 log.cc:824] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:09:12 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:12.668746 11888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7: No bootstrap required, opened a new log
I20260809 13:09:12.675294 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:12.654241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.10.181.190:43363)
0809 13:09:12.655013 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:12.655026 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:12.655070 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:09:12.656735 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:12.656759 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:12.657162 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:12.657520 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:12.659266 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:12.660548 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:12.662268 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:12.662924 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:12.663052 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:12.667832 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:12.667840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:12.667854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:12.667892 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:12.671216 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:12.671990 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:12.671996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:12.672005 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:12.672182 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:12.672543 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:12.672546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:12.672548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:12.673093 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:12.673317 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:12.673887 (+ 570us) server_negotiation.cc:299] Negotiation successful
0809 13:09:12.674273 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":366,"threads_started":1}
I20260809 13:09:12.677614 11888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } }
I20260809 13:09:12.677950 11888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:12.678056 11888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2777ba45e645d78ce2f28d2e1576a7, State: Initialized, Role: FOLLOWER
I20260809 13:09:12.678550 11888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } }
I20260809 13:09:12.678767 11888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:12.678893 11888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:12.679050 11888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:12.681193 11888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } }
I20260809 13:09:12.681706 11888 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2777ba45e645d78ce2f28d2e1576a7; no voters:
I20260809 13:09:12.682238 11888 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:12.682367 11896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:12.682986 11896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d2777ba45e645d78ce2f28d2e1576a7, State: Running, Role: LEADER
I20260809 13:09:12.683554 11896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } }
I20260809 13:09:12.684314 11888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:12.686615 11902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2777ba45e645d78ce2f28d2e1576a7. Latest consensus state: current_term: 1 leader_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } } }
I20260809 13:09:12.686596 11901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2777ba45e645d78ce2f28d2e1576a7" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43363 } } }
I20260809 13:09:12.687083 11901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:12.687096 11902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:12.687973 11909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:12.692610 11909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:12.700280 11909 catalog_manager.cc:1347] Generated new cluster ID: 5cd91dd7ec4d4b6c8877507eeadf95aa
I20260809 13:09:12.700390 11909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:12.716050 11909 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:12.717486 11909 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:12.724395 11909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2777ba45e645d78ce2f28d2e1576a7: Generated new TSK 0
I20260809 13:09:12.725180 11909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:12.770495 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:13.100731 11926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:13.101006 11926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:13.101145 11926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:13.108314 11926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:13.108441 11926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:13.108597 11926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:13.118695 11926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:13.120052 11926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:13.121426 11926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:13.138113 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:13.138178 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:13.138206 11942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:13.139472 11926 server_base.cc:1034] running on GCE node
I20260809 13:09:13.140096 11926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:13.141029 11926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:13.142231 11926 hybrid_clock.cc:648] HybridClock initialized: now 1786280953142172 us; error 47 us; skew 500 ppm
I20260809 13:09:13.142522 11926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:13 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280953, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:13.145658 11926 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:13.147167 11926 webserver.cc:460] Webserver started at http://127.10.181.129:40713/ using document root <none> and password file <none>
I20260809 13:09:13.147858 11926 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:13.147980 11926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:13.148293 11926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:13.151338 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "ac7832c082c44a30b69917e54eaa6939"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "520ba4ab1b6d9cc8493e34ac7c51a5a6"
server_key_iv: "825ea2fded34caa9fe766862e5dd5e47"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.152117 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "ac7832c082c44a30b69917e54eaa6939"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "520ba4ab1b6d9cc8493e34ac7c51a5a6"
server_key_iv: "825ea2fded34caa9fe766862e5dd5e47"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.158424 11926 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 13:09:13.162143 11953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.163365 11926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:13.163523 11926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "ac7832c082c44a30b69917e54eaa6939"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "520ba4ab1b6d9cc8493e34ac7c51a5a6"
server_key_iv: "825ea2fded34caa9fe766862e5dd5e47"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.163726 11926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:13.189930 11926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:13.193854 11926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:13.194123 11926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:13.195072 11926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:13.196384 11926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:13.196457 11926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.196599 11926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:13.196650 11926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.226613 11926 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:35531
I20260809 13:09:13.226696 12118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:35531 every 8 connection(s)
I20260809 13:09:13.227885 11926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:13.230223 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11926
I20260809 13:09:13.230454 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:13.231289 10966 external_mini_cluster.cc:1428] Setting key 78218e813147b6e263141e86567b8f8c
Aug 09 13:09:13 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280953, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:13.247922 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:13.230467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:59693 (local address 127.10.181.190:43363)
0809 13:09:13.230934 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:13.230939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:13.232229 (+ 1290us) server_negotiation.cc:407] Connection header received
0809 13:09:13.234380 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:13.234385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:13.234461 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:13.234550 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:13.236049 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.236767 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.238324 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.238737 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.238851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:13.242588 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:13.242593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:13.242596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:13.242629 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:13.244965 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:13.245706 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:13.245710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:13.245713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:13.245828 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:13.246245 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:13.246248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:13.246250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:13.246508 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:13.246653 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:13.247498 (+ 845us) server_negotiation.cc:299] Negotiation successful
0809 13:09:13.247675 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
I20260809 13:09:13.249629 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:13.231166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43363 (local address 127.10.181.129:59693)
0809 13:09:13.231993 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:13.232049 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:13.234125 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:13.234794 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:13.234803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:13.235325 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:13.235885 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:13.235902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.236914 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.236919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:13.238186 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:13.238200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.238970 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.238977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:13.239208 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:13.239847 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:13.239875 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:13.242374 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:13.245100 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:13.245109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:13.245125 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:13.245586 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:13.245933 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:13.245938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:13.245942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:13.246131 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:13.246664 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:13.246673 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:13.247111 (+ 438us) client_negotiation.cc:769] Sending connection context
0809 13:09:13.247421 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 13:09:13.248188 (+ 767us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
I20260809 13:09:13.251179 12119 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43363
I20260809 13:09:13.251457 12119 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:13.252151 12119 heartbeater.cc:511] Master 127.10.181.190:43363 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
I20260809 13:09:13.254454 11833 ts_manager.cc:194] Registered new tserver with Master: ac7832c082c44a30b69917e54eaa6939 (127.10.181.129:35531)
I20260809 13:09:13.256338 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:59693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:13.275969 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:13.597920 12127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:13.598222 12127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:13.598369 12127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:13.605495 12127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:13.605611 12127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:13.605770 12127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:13.615424 12127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:13.616771 12127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:13.618108 12127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:13.630867 12136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:13.630898 12137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:13.631071 12140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:13.632058 12127 server_base.cc:1034] running on GCE node
I20260809 13:09:13.632668 12127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:13.633628 12127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:13.634831 12127 hybrid_clock.cc:648] HybridClock initialized: now 1786280953634777 us; error 42 us; skew 500 ppm
I20260809 13:09:13.635142 12127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:13 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280953, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:13.638181 12127 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:13.639663 12127 webserver.cc:460] Webserver started at http://127.10.181.130:34257/ using document root <none> and password file <none>
I20260809 13:09:13.640360 12127 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:13.640479 12127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:13.640789 12127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:13.643882 12127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "79ae676f74874da38f787acdc77793a3"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "41fe0b7acec5254adf7a5b374c15a1f8"
server_key_iv: "7d3183d7134f7887ef4b311351e2f219"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.644659 12127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "79ae676f74874da38f787acdc77793a3"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "41fe0b7acec5254adf7a5b374c15a1f8"
server_key_iv: "7d3183d7134f7887ef4b311351e2f219"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.649842 12127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 13:09:13.653374 12150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.654831 12127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:13.655014 12127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "79ae676f74874da38f787acdc77793a3"
format_stamp: "Formatted at 2026-08-09 13:09:13 on dist-test-slave-42z9"
server_key: "41fe0b7acec5254adf7a5b374c15a1f8"
server_key_iv: "7d3183d7134f7887ef4b311351e2f219"
server_key_version: "encryptionkey@0"
I20260809 13:09:13.655211 12127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:13.674603 12127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:13.679219 12127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:13.679514 12127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:13.680509 12127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:13.681892 12127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:13.681980 12127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.682106 12127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:13.682163 12127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:13.712761 12127 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:41269
I20260809 13:09:13.712837 12334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:41269 every 8 connection(s)
I20260809 13:09:13.714093 12127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:13.723351 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12127
I20260809 13:09:13.723565 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:13.724164 10966 external_mini_cluster.cc:1428] Setting key 6bd42150e4ef0f60f550711d663f8bd2
Aug 09 13:09:13 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280953, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:13.734952 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:13.716634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59793 (local address 127.10.181.190:43363)
0809 13:09:13.716828 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:13.716833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:13.718040 (+ 1207us) server_negotiation.cc:407] Connection header received
0809 13:09:13.719926 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:13.719932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:13.720019 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:13.720111 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:13.721684 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.722430 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.724584 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.725152 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.725276 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:13.728861 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:13.728866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:13.728870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:13.728904 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:13.731885 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:13.732607 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:13.732612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:13.732615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:13.732779 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:13.733160 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:13.733164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:13.733167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:13.733560 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:13.733676 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:13.734532 (+ 856us) server_negotiation.cc:299] Negotiation successful
0809 13:09:13.734713 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:09:13.736375 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:13.717136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43363 (local address 127.10.181.130:59793)
0809 13:09:13.717874 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:13.717913 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:13.719647 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:13.720370 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:13.720379 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:13.720899 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:13.721502 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:13.721523 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.722590 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.722594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:13.724404 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:13.724420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:13.725387 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:13.725395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:13.725656 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:13.726289 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:13.726318 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:13.728630 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:13.732028 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:13.732036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:13.732051 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:13.732479 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:13.732869 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:13.732872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:13.732875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:13.733052 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:13.733699 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:13.733707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:13.734159 (+ 452us) client_negotiation.cc:769] Sending connection context
0809 13:09:13.734474 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 13:09:13.735024 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
I20260809 13:09:13.737900 12336 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43363
I20260809 13:09:13.738211 12336 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:13.738941 12336 heartbeater.cc:511] Master 127.10.181.190:43363 requested a full tablet report, sending...
I20260809 13:09:13.740484 11833 ts_manager.cc:194] Registered new tserver with Master: 79ae676f74874da38f787acdc77793a3 (127.10.181.130:41269)
I20260809 13:09:13.741626 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:59793
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:13.768757 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:14.091684 12349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:14.091964 12349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:14.092070 12349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:14.099141 12349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:14.099244 12349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:14.099366 12349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:14.109030 12349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:14.110349 12349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:14.111693 12349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:14.124424 12359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:14.124351 12361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:14.124328 12364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:14.125113 12349 server_base.cc:1034] running on GCE node
I20260809 13:09:14.125725 12349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:14.126595 12349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:14.127777 12349 hybrid_clock.cc:648] HybridClock initialized: now 1786280954127722 us; error 51 us; skew 500 ppm
I20260809 13:09:14.128074 12349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:14.131229 12349 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:14.132725 12349 webserver.cc:460] Webserver started at http://127.10.181.131:32859/ using document root <none> and password file <none>
I20260809 13:09:14.133405 12349 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:14.133559 12349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:14.133865 12349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:14.136767 12349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "163ced0532184a6aab52abcb157443f1"
format_stamp: "Formatted at 2026-08-09 13:09:14 on dist-test-slave-42z9"
server_key: "cce04bb4d86a533e820ddd20837174d9"
server_key_iv: "c0a0a10049f6e70abea966050a05b57a"
server_key_version: "encryptionkey@0"
I20260809 13:09:14.137598 12349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "163ced0532184a6aab52abcb157443f1"
format_stamp: "Formatted at 2026-08-09 13:09:14 on dist-test-slave-42z9"
server_key: "cce04bb4d86a533e820ddd20837174d9"
server_key_iv: "c0a0a10049f6e70abea966050a05b57a"
server_key_version: "encryptionkey@0"
I20260809 13:09:14.142504 12349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:09:14.145874 12373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:14.147107 12349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:09:14.147279 12349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "163ced0532184a6aab52abcb157443f1"
format_stamp: "Formatted at 2026-08-09 13:09:14 on dist-test-slave-42z9"
server_key: "cce04bb4d86a533e820ddd20837174d9"
server_key_iv: "c0a0a10049f6e70abea966050a05b57a"
server_key_version: "encryptionkey@0"
I20260809 13:09:14.147455 12349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:14.171937 12349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:14.175920 12349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:14.176182 12349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:14.177120 12349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:14.178458 12349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:14.178556 12349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:14.178655 12349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:14.178699 12349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:14.208422 12349 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:40751
I20260809 13:09:14.208492 12542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:40751 every 8 connection(s)
I20260809 13:09:14.209746 12349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:14.215929 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12349
I20260809 13:09:14.216101 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:14.216518 10966 external_mini_cluster.cc:1428] Setting key e6ca619ef2407914a827f70aa95b5ef3
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:14.228986 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.212229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:44791 (local address 127.10.181.190:43363)
0809 13:09:14.212419 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.212424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.213583 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 13:09:14.215493 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.215498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.215615 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.215707 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.217347 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.218367 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.219951 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.220357 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.220467 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.223667 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.223671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.223675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.223707 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.226085 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.226775 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.226780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.226783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.226908 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.227322 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.227325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.227328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.227622 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.227755 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.228419 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.228654 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:09:14.230005 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.212687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43363 (local address 127.10.181.131:44791)
0809 13:09:14.213390 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.213458 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.215226 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.215951 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.215959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.216506 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:14.217171 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.217198 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.218513 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.218517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.219816 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.219828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.220573 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.220580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.220797 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.221354 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:14.221375 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:14.223481 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:14.226237 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:14.226245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:14.226259 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:14.226665 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:14.227016 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:14.227020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:14.227024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:14.227214 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:14.227755 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:14.227763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:14.228064 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.228297 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.228792 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":290,"threads_started":1}
I20260809 13:09:14.231794 12543 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43363
I20260809 13:09:14.232064 12543 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:14.232759 12543 heartbeater.cc:511] Master 127.10.181.190:43363 requested a full tablet report, sending...
I20260809 13:09:14.234112 11833 ts_manager.cc:194] Registered new tserver with Master: 163ced0532184a6aab52abcb157443f1 (127.10.181.131:40751)
I20260809 13:09:14.234910 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:44791
I20260809 13:09:14.243690 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:14.259269 12119 heartbeater.cc:503] Master 127.10.181.190:43363 was elected leader, sending a full tablet report...
I20260809 13:09:14.261602 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.249634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35266 (local address 127.10.181.190:43363)
0809 13:09:14.249838 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.249844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.250277 (+ 433us) server_negotiation.cc:407] Connection header received
0809 13:09:14.250414 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.250418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.250480 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.250557 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.251956 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.252692 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.253905 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.254293 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.254383 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.256416 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.256421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.256424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.256454 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.258868 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.259517 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.259523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.259526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.259722 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.260091 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.260097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.260101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.260467 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.260710 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.261059 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.261280 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:09:14.267352 11833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:14.274009 11833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:14.297180 12563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.286133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:35531 (local address 127.0.0.1:40300)
0809 13:09:14.286831 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.286883 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.287105 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.289037 (+ 1932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.289042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.289066 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:14.290095 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.290108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.291980 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.291986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.294346 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.294359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.296088 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.296095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.296322 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.296388 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.296515 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.296677 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":24,"thread_start_us":1196,"threads_started":1}
I20260809 13:09:14.297248 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.287242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44512 (local address 127.10.181.130:41269)
0809 13:09:14.289973 (+ 2731us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.289982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.290009 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:09:14.290113 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.290132 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.290342 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.290539 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:14.291353 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.292823 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.295196 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.296227 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.296389 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.296561 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.296767 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.296941 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2266,"thread_start_us":2172,"threads_started":1}
I20260809 13:09:14.297858 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.286532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.10.181.129:35531)
0809 13:09:14.288180 (+ 1648us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.288186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.288210 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:09:14.288301 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.288309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.288579 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.288780 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:14.290286 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.291836 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.294499 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.295853 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.296060 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.296227 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.297490 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.297631 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":1347,"threads_started":1}
I20260809 13:09:14.297231 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.286647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41269 (local address 127.0.0.1:44512)
0809 13:09:14.288670 (+ 2023us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.288688 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.288793 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.290772 (+ 1979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.290777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.290803 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:14.291174 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.291186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.292966 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.292971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.295052 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.295065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.296432 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.296438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.296587 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.296621 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.296718 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.296846 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1894,"thread_start_us":2606,"threads_started":1}
I20260809 13:09:14.299655 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.288129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40751 (local address 127.0.0.1:42910)
0809 13:09:14.289557 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.289573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.289702 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.292330 (+ 2628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.292335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.292358 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:14.292807 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.292817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.294814 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.294820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.296787 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.296800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.298878 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.298884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.299057 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.299089 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.299173 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.299294 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1284,"mutex_wait_us":66,"thread_start_us":864,"threads_started":1}
I20260809 13:09:14.299757 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.288753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.10.181.131:40751)
0809 13:09:14.291862 (+ 3109us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.291870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.291900 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:09:14.291991 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.291996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.292153 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.292310 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:14.293033 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.294669 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.296960 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.298684 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.298871 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.299028 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.299244 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.299425 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2944,"thread_start_us":2803,"threads_started":1}
I20260809 13:09:14.301676 12019 tablet_service.cc:1467] Processing CreateTablet for tablet a50feaf220ee44fd825aa74bd5a6c4d8 (DEFAULT_TABLE table=test-table [id=804155ec87fb435583faf61ec0433db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:14.301678 12231 tablet_service.cc:1467] Processing CreateTablet for tablet a50feaf220ee44fd825aa74bd5a6c4d8 (DEFAULT_TABLE table=test-table [id=804155ec87fb435583faf61ec0433db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:14.303141 12231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50feaf220ee44fd825aa74bd5a6c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:14.303145 12019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50feaf220ee44fd825aa74bd5a6c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:14.304967 12439 tablet_service.cc:1467] Processing CreateTablet for tablet a50feaf220ee44fd825aa74bd5a6c4d8 (DEFAULT_TABLE table=test-table [id=804155ec87fb435583faf61ec0433db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:14.306353 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a50feaf220ee44fd825aa74bd5a6c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:14.314127 12569 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Bootstrap starting.
I20260809 13:09:14.317313 12570 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Bootstrap starting.
I20260809 13:09:14.317829 12569 tablet_bootstrap.cc:654] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:14.319207 12569 log.cc:824] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:14.320209 12577 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Bootstrap starting.
I20260809 13:09:14.320905 12570 tablet_bootstrap.cc:654] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:14.322042 12570 log.cc:824] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:14.323310 12569 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: No bootstrap required, opened a new log
I20260809 13:09:14.323401 12577 tablet_bootstrap.cc:654] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:14.323645 12569 ts_tablet_manager.cc:1397] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 13:09:14.324471 12577 log.cc:824] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:14.325701 12570 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: No bootstrap required, opened a new log
I20260809 13:09:14.325968 12570 ts_tablet_manager.cc:1397] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 13:09:14.327517 12577 tablet_bootstrap.cc:492] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: No bootstrap required, opened a new log
I20260809 13:09:14.327780 12577 ts_tablet_manager.cc:1397] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 13:09:14.334020 12569 raft_consensus.cc:348] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.334502 12569 raft_consensus.cc:374] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:14.334635 12569 raft_consensus.cc:729] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ae676f74874da38f787acdc77793a3, State: Initialized, Role: FOLLOWER
I20260809 13:09:14.334870 12577 raft_consensus.cc:348] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.335374 12577 raft_consensus.cc:374] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:14.335522 12577 raft_consensus.cc:729] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163ced0532184a6aab52abcb157443f1, State: Initialized, Role: FOLLOWER
I20260809 13:09:14.335413 12569 consensus_queue.cc:260] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.336124 12577 consensus_queue.cc:260] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.336645 12336 heartbeater.cc:503] Master 127.10.181.190:43363 was elected leader, sending a full tablet report...
I20260809 13:09:14.336850 12570 raft_consensus.cc:348] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.337329 12569 ts_tablet_manager.cc:1428] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Time spent starting tablet: real 0.014s user 0.002s sys 0.011s
I20260809 13:09:14.337332 12570 raft_consensus.cc:374] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:14.337431 12570 raft_consensus.cc:729] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7832c082c44a30b69917e54eaa6939, State: Initialized, Role: FOLLOWER
I20260809 13:09:14.338303 12543 heartbeater.cc:503] Master 127.10.181.190:43363 was elected leader, sending a full tablet report...
I20260809 13:09:14.338171 12570 consensus_queue.cc:260] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.338862 12577 ts_tablet_manager.cc:1428] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260809 13:09:14.341125 12570 ts_tablet_manager.cc:1428] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 13:09:14.461313 12546 tablet.cc:2369] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:09:14.466054 12340 tablet.cc:2369] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:09:14.480064 12120 tablet.cc:2369] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:09:14.578161 12584 raft_consensus.cc:482] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:14.578508 12584 raft_consensus.cc:504] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.580302 12584 leader_election.cc:288] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac7832c082c44a30b69917e54eaa6939 (127.10.181.129:35531), 79ae676f74874da38f787acdc77793a3 (127.10.181.130:41269)
I20260809 13:09:14.586501 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.580699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:35531 (local address 127.10.181.131:58787)
0809 13:09:14.581502 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.581519 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.581646 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.582042 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.582047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.582082 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:14.582291 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.582302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.583250 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.583255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.584595 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.584608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.585794 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.585800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.585973 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.585998 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.586079 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.586207 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 13:09:14.586526 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.581069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:58787 (local address 127.10.181.129:35531)
0809 13:09:14.581336 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.581343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.581601 (+ 258us) server_negotiation.cc:407] Connection header received
0809 13:09:14.581825 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.581830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.581913 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.582009 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:14.582417 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.583123 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.584752 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.585650 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.585793 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.585869 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.586128 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.586249 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 13:09:14.587270 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.581056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:37271 (local address 127.10.181.130:41269)
0809 13:09:14.581337 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.581343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.582216 (+ 873us) server_negotiation.cc:407] Connection header received
0809 13:09:14.582349 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.582353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.582411 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.582489 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:14.583036 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.583737 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.585513 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.586342 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.586481 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.586548 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.586895 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.587019 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 13:09:14.587312 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.580960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41269 (local address 127.10.181.131:37271)
0809 13:09:14.582109 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:14.582126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:14.582248 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:14.582529 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:14.582534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:14.582559 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:14.582872 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.582880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.583906 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.583911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.585329 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:14.585342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.586549 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.586556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:14.586724 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.586753 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 13:09:14.586845 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:09:14.586965 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"thread_start_us":830,"threads_started":1}
I20260809 13:09:14.587604 12050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50feaf220ee44fd825aa74bd5a6c4d8" candidate_uuid: "163ced0532184a6aab52abcb157443f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7832c082c44a30b69917e54eaa6939" is_pre_election: true
I20260809 13:09:14.588232 12050 raft_consensus.cc:2454] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163ced0532184a6aab52abcb157443f1 in term 0.
I20260809 13:09:14.588294 12258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50feaf220ee44fd825aa74bd5a6c4d8" candidate_uuid: "163ced0532184a6aab52abcb157443f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ae676f74874da38f787acdc77793a3" is_pre_election: true
I20260809 13:09:14.588829 12258 raft_consensus.cc:2454] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 163ced0532184a6aab52abcb157443f1 in term 0.
I20260809 13:09:14.589062 12376 leader_election.cc:302] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163ced0532184a6aab52abcb157443f1, ac7832c082c44a30b69917e54eaa6939; no voters:
I20260809 13:09:14.589501 12584 raft_consensus.cc:2781] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:14.589654 12584 raft_consensus.cc:482] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:14.589757 12584 raft_consensus.cc:3037] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:14.591940 12584 raft_consensus.cc:504] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.592567 12584 leader_election.cc:288] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [CANDIDATE]: Term 1 election: Requested vote from peers ac7832c082c44a30b69917e54eaa6939 (127.10.181.129:35531), 79ae676f74874da38f787acdc77793a3 (127.10.181.130:41269)
I20260809 13:09:14.592978 12050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50feaf220ee44fd825aa74bd5a6c4d8" candidate_uuid: "163ced0532184a6aab52abcb157443f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac7832c082c44a30b69917e54eaa6939"
I20260809 13:09:14.593060 12258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a50feaf220ee44fd825aa74bd5a6c4d8" candidate_uuid: "163ced0532184a6aab52abcb157443f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ae676f74874da38f787acdc77793a3"
I20260809 13:09:14.593197 12050 raft_consensus.cc:3037] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:14.593255 12258 raft_consensus.cc:3037] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:14.596356 12258 raft_consensus.cc:2454] T a50feaf220ee44fd825aa74bd5a6c4d8 P 79ae676f74874da38f787acdc77793a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163ced0532184a6aab52abcb157443f1 in term 1.
I20260809 13:09:14.596356 12050 raft_consensus.cc:2454] T a50feaf220ee44fd825aa74bd5a6c4d8 P ac7832c082c44a30b69917e54eaa6939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 163ced0532184a6aab52abcb157443f1 in term 1.
I20260809 13:09:14.596761 12374 leader_election.cc:302] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 163ced0532184a6aab52abcb157443f1, 79ae676f74874da38f787acdc77793a3; no voters:
I20260809 13:09:14.597028 12584 raft_consensus.cc:2781] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:14.597496 12584 raft_consensus.cc:686] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [term 1 LEADER]: Becoming Leader. State: Replica: 163ced0532184a6aab52abcb157443f1, State: Running, Role: LEADER
I20260809 13:09:14.597963 12584 consensus_queue.cc:237] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } }
I20260809 13:09:14.602747 11832 catalog_manager.cc:5495] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 163ced0532184a6aab52abcb157443f1 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "163ced0532184a6aab52abcb157443f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac7832c082c44a30b69917e54eaa6939" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "163ced0532184a6aab52abcb157443f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79ae676f74874da38f787acdc77793a3" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41269 } health_report { overall_health: UNKNOWN } } }
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:14.667505 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.652004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.10.181.129:35531)
0809 13:09:14.652238 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.652245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.652684 (+ 439us) server_negotiation.cc:407] Connection header received
0809 13:09:14.652863 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.652867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.652944 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.653026 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.654955 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.655732 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.657565 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.657962 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.658078 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.662143 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.662160 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.662178 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.662212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.664788 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.665435 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.665463 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.665467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.665626 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.665986 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.665989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.665991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.666359 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.666669 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.666949 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.667168 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:14.717949 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.704432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.10.181.129:35531)
0809 13:09:14.704635 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.704642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.704950 (+ 308us) server_negotiation.cc:407] Connection header received
0809 13:09:14.705128 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.705132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.705202 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.705289 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.706659 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.707409 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.708740 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.709165 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.709279 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.712983 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.712988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.712991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.713026 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.715514 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.716080 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.716084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.716087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.716218 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.716579 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.716583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.716585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.716893 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.717135 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.717430 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.717665 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:14.767369 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.753912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.10.181.129:35531)
0809 13:09:14.754096 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.754103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.754452 (+ 349us) server_negotiation.cc:407] Connection header received
0809 13:09:14.754635 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.754639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.754710 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.754794 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.756139 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.756857 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.758126 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.758560 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.758682 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.762494 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.762499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.762503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.762539 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.764860 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.765494 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.765499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.765501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.765632 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.765971 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.765975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.765977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.766279 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.766533 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.766800 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.767027 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 13:09:14.769236 11992 tablet_service.cc:2908] Rejecting scan request for tablet a50feaf220ee44fd825aa74bd5a6c4d8: Uninitialized: clean time has not yet been initialized
I20260809 13:09:14.772105 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11926
I20260809 13:09:14.783552 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.772967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.10.181.129:35531)
0809 13:09:14.773163 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.773170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.773474 (+ 304us) server_negotiation.cc:407] Connection header received
0809 13:09:14.773645 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.773648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.773721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.773796 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.774883 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.775531 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.776609 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.776996 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.777106 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.779172 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.779176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.779179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.779206 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.781483 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.782066 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.782078 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.782081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.782187 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.782438 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.782441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.782442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.782703 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.782913 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.783108 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.783269 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:09:14.784183 12113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:14.872803 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11926
I20260809 13:09:14.900240 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12127
Aug 09 13:09:14 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:14.914039 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:14.901025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.10.181.130:41269)
0809 13:09:14.901216 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:14.901220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:14.901240 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:09:14.901350 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:14.901355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:14.901427 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:14.901542 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:14.902646 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.903343 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.904520 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:14.904883 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:14.904971 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:14.908802 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:14.908821 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:14.908842 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:14.908875 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:14.911384 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.912023 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.912031 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.912033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.912165 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:14.912520 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:14.912524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:14.912526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:14.912898 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:14.913189 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:14.913508 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 13:09:14.913733 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:09:14.914656 12326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:15.001295 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12127
W20260809 13:09:15.025574 12374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.130:41269 (error 108)
I20260809 13:09:15.025754 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12349
I20260809 13:09:15.027050 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:15.026525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:35531 (local address 127.10.181.131:38243)
0809 13:09:15.026725 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:15.026860 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:35531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 13:09:15.027686 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:15.027145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41269 (local address 127.10.181.131:34617)
0809 13:09:15.027404 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:15.027492 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.130:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":34}
W20260809 13:09:15.028050 12374 consensus_peers.cc:487] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 -> Peer 79ae676f74874da38f787acdc77793a3 (127.10.181.130:41269): Couldn't send request to peer 79ae676f74874da38f787acdc77793a3. Status: Network error: Client connection negotiation failed: client connection to 127.10.181.130:41269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 13:09:15.028225 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:15.027790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:35531 (local address 127.10.181.131:45823)
0809 13:09:15.028009 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:15.028072 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:35531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":39}
W20260809 13:09:15.028398 12376 consensus_peers.cc:487] T a50feaf220ee44fd825aa74bd5a6c4d8 P 163ced0532184a6aab52abcb157443f1 -> Peer ac7832c082c44a30b69917e54eaa6939 (127.10.181.129:35531): Couldn't send request to peer ac7832c082c44a30b69917e54eaa6939. Status: Network error: Client connection negotiation failed: client connection to 127.10.181.129:35531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 09 13:09:15 dist-test-slave-42z9 krb5kdc[11770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:15.040628 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:15.026723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.10.181.131:40751)
0809 13:09:15.027470 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:15.027476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:15.027500 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:09:15.027559 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:15.027564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:15.027649 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:15.027770 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:15.029250 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:15.029974 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:15.031201 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:15.031581 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:15.031673 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:15.035399 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:15.035411 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:15.035429 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:15.035462 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:15.038077 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:15.038698 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:15.038705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:15.038708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:15.038850 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:15.039241 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:15.039244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:15.039246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:15.039603 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:15.039850 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:15.040073 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 13:09:15.040268 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":548,"threads_started":1}
I20260809 13:09:15.041299 12536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:15.129784 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12349
I20260809 13:09:15.155942 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11790
I20260809 13:09:15.157132 11879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:15.266759 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11790
2026-08-09T13:09:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4412 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:15 dist-test-slave-42z9 krb5kdc[12625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:15 dist-test-slave-42z9 krb5kdc[12625](info): set up 2 sockets
Aug 09 13:09:15 dist-test-slave-42z9 krb5kdc[12625](info): commencing operation
krb5kdc: starting...
W20260809 13:09:16.453245 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:16 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:16.637686 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46639
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:16.958392 12639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:16.958678 12639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:16.958807 12639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:16.965816 12639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:16.965919 12639 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 13:09:16.965988 12639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:16.966085 12639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:16.966148 12639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:16.966229 12639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:16.975631 12639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46639
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:16.976903 12639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:16.978236 12639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:16.990833 12653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:16.990875 12656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:16.991039 12658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:16.991997 12639 server_base.cc:1034] running on GCE node
I20260809 13:09:16.992751 12639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:16.994125 12639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:16.995339 12639 hybrid_clock.cc:648] HybridClock initialized: now 1786280956995273 us; error 32 us; skew 500 ppm
I20260809 13:09:16.995669 12639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:16 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280956, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:16.998703 12639 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:17.000144 12639 webserver.cc:460] Webserver started at http://127.10.181.190:35829/ using document root <none> and password file <none>
I20260809 13:09:17.000816 12639 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:17.000912 12639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:17.001237 12639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:17.004207 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "4a25b37de833421c87872e3b2bfe170e"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "32f282b341a109ff31b944c2fca67f11"
server_key_iv: "c8a13aa65c8d8b16e6f6440b05f60e22"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.004952 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "4a25b37de833421c87872e3b2bfe170e"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "32f282b341a109ff31b944c2fca67f11"
server_key_iv: "c8a13aa65c8d8b16e6f6440b05f60e22"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.009883 12639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 13:09:17.013149 12668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:17.014386 12639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:17.014547 12639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "4a25b37de833421c87872e3b2bfe170e"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "32f282b341a109ff31b944c2fca67f11"
server_key_iv: "c8a13aa65c8d8b16e6f6440b05f60e22"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.014736 12639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:17.030645 12639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:17.034667 12639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:17.034919 12639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:17.055351 12639 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46639
I20260809 13:09:17.055372 12761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46639 every 8 connection(s)
I20260809 13:09:17.056676 12639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:17.060426 12762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:17.062608 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12639
I20260809 13:09:17.062806 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:17.063436 10966 external_mini_cluster.cc:1428] Setting key 18d8a8996b8b23d51b936ee8d68c553b
I20260809 13:09:17.072954 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Bootstrap starting.
I20260809 13:09:17.076378 12762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:17.077328 12762 log.cc:824] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:09:17 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:17.080116 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: No bootstrap required, opened a new log
I20260809 13:09:17.086457 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:17.065332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.10.181.190:46639)
0809 13:09:17.066168 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:17.066182 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:17.066233 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:09:17.068555 (+ 2322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:17.068583 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:17.069050 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:17.069512 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:17.070737 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.072608 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.073826 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.074873 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.075060 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:17.078998 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:17.079005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:17.079019 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:17.079051 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:17.082172 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.082963 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.082969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.082975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.083160 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.083580 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.083585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.083587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.084112 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:17.084321 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:17.084822 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 13:09:17.085271 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":365,"threads_started":1}
I20260809 13:09:17.089167 12762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } }
I20260809 13:09:17.089618 12762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:17.089711 12762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a25b37de833421c87872e3b2bfe170e, State: Initialized, Role: FOLLOWER
I20260809 13:09:17.090243 12762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } }
I20260809 13:09:17.090471 12762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:17.090606 12762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:17.090751 12762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:17.092906 12762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } }
I20260809 13:09:17.093360 12762 leader_election.cc:302] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a25b37de833421c87872e3b2bfe170e; no voters:
I20260809 13:09:17.093977 12762 leader_election.cc:288] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:17.094076 12771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:17.094704 12771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [term 1 LEADER]: Becoming Leader. State: Replica: 4a25b37de833421c87872e3b2bfe170e, State: Running, Role: LEADER
I20260809 13:09:17.095191 12771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } }
I20260809 13:09:17.095889 12762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:17.097775 12773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a25b37de833421c87872e3b2bfe170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } } }
I20260809 13:09:17.098155 12773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:17.098670 12774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a25b37de833421c87872e3b2bfe170e. Latest consensus state: current_term: 1 leader_uuid: "4a25b37de833421c87872e3b2bfe170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a25b37de833421c87872e3b2bfe170e" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46639 } } }
I20260809 13:09:17.099022 12774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:17.100257 12780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:17.105280 12780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:17.113940 12780 catalog_manager.cc:1347] Generated new cluster ID: 60c7e1a952154cac806d28156f9b91c9
I20260809 13:09:17.114044 12780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:17.161375 12780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:17.163259 12780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:17.179431 12780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Generated new TSK 0
I20260809 13:09:17.180493 12780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:17.233592 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46639
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:17.560761 12801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:17.561014 12801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:17.561151 12801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:17.568250 12801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:17.568360 12801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:17.568490 12801 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:17.568593 12801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:17.578223 12801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:46639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:17.579509 12801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:17.580801 12801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:17.593544 12807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:17.593688 12806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:17.593683 12810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:17.594303 12801 server_base.cc:1034] running on GCE node
I20260809 13:09:17.594848 12801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:17.595713 12801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:17.596933 12801 hybrid_clock.cc:648] HybridClock initialized: now 1786280957596865 us; error 32 us; skew 500 ppm
I20260809 13:09:17.597213 12801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:17 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280957, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:17.603633 12801 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:17.605034 12801 webserver.cc:460] Webserver started at http://127.10.181.129:36429/ using document root <none> and password file <none>
I20260809 13:09:17.605762 12801 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:17.605867 12801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:17.606180 12801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:17.609045 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "5e93237324174f569cfc9b4c9654bfdc"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "121981dd2c3e836c4d218c4a4dcb2b65"
server_key_iv: "667a2b27350e59424808b581d3b35eb5"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.609850 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "5e93237324174f569cfc9b4c9654bfdc"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "121981dd2c3e836c4d218c4a4dcb2b65"
server_key_iv: "667a2b27350e59424808b581d3b35eb5"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.614831 12801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 13:09:17.618335 12821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:17.619653 12801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:17.619782 12801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "5e93237324174f569cfc9b4c9654bfdc"
format_stamp: "Formatted at 2026-08-09 13:09:17 on dist-test-slave-42z9"
server_key: "121981dd2c3e836c4d218c4a4dcb2b65"
server_key_iv: "667a2b27350e59424808b581d3b35eb5"
server_key_version: "encryptionkey@0"
I20260809 13:09:17.619963 12801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:17.638478 12801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:17.642226 12801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:17.642469 12801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:17.645236 12801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:17.645314 12801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:17.645468 12801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:17.645543 12801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 13:09:17 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280957, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:17.667758 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:17.645714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:43151 (local address 127.10.181.190:46639)
0809 13:09:17.646444 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:17.646450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:17.647416 (+ 966us) server_negotiation.cc:407] Connection header received
0809 13:09:17.649424 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:17.649430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:17.649549 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:17.649673 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:17.651784 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.652836 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.655645 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.656124 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.656246 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:17.661103 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:17.661108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:17.661111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:17.661146 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:17.664040 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.665034 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.665040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.665043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.665198 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.665841 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.665847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.665851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.666267 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:17.666433 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:17.667089 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 13:09:17.667335 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":490,"threads_started":1}
I20260809 13:09:17.668533 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:17.646352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.129:43151)
0809 13:09:17.647238 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:17.647285 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:17.649158 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:17.649977 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:17.649990 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:17.650711 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:17.651590 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:17.651622 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.653045 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.653051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:17.655479 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:17.655498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.656404 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.656416 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:17.656827 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:17.657663 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:17.657705 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:17.660874 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:17.664210 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:17.664217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:17.664229 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:17.664867 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:17.665338 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:17.665341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:17.665344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:17.665682 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:17.666397 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:17.666402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:17.666748 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 13:09:17.666977 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 13:09:17.667358 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":370,"threads_started":1}
I20260809 13:09:17.688371 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:17.675588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.129:45107)
0809 13:09:17.676164 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:17.676180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:17.676306 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:17.676604 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:17.676608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:17.677060 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:17.677356 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:17.677365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.678666 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.678671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:17.679932 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:17.679949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.680807 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.680814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:17.680961 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:17.681542 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:17.681560 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:17.682534 (+ 974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:17.685334 (+ 2800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:17.685340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:17.685344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:17.685820 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:17.686287 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:17.686295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:17.686298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:17.686473 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:17.687090 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:17.687095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:17.687333 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 13:09:17.687511 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 13:09:17.687840 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":311,"threads_started":1}
I20260809 13:09:17.689096 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:17.675498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:45107 (local address 127.10.181.190:46639)
0809 13:09:17.675657 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:17.675661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:17.676260 (+ 599us) server_negotiation.cc:407] Connection header received
0809 13:09:17.676415 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:17.676421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:17.676484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:17.676563 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:17.677526 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.678463 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.680090 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:17.680610 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:17.680720 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:17.682721 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:17.682726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:17.682729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:17.682760 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:17.685214 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.685964 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.685971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.685975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.686141 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:17.686598 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:17.686603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:17.686607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:17.686977 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:17.687119 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:17.688697 (+ 1578us) server_negotiation.cc:299] Negotiation successful
0809 13:09:17.688861 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 13:09:17.692570 12801 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:34751
I20260809 13:09:17.692597 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:34751 every 8 connection(s)
I20260809 13:09:17.693990 12801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:17.696072 13011 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46639
I20260809 13:09:17.696300 13011 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:17.696959 13011 heartbeater.cc:511] Master 127.10.181.190:46639 requested a full tablet report, sending...
I20260809 13:09:17.698879 12698 ts_manager.cc:194] Registered new tserver with Master: 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751)
I20260809 13:09:17.700462 12698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:43151
I20260809 13:09:17.703974 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12801
I20260809 13:09:17.704169 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:17.704782 10966 external_mini_cluster.cc:1428] Setting key 3833abf70614a946670ba66067e1014f
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:17.746528 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46639
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:18.078527 13017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:18.078781 13017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:18.078917 13017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:18.086040 13017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:18.086138 13017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:18.086261 13017 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:18.086364 13017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:18.095716 13017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:46639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:18.096912 13017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:18.098215 13017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:18.110625 13029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:18.110673 13031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:18.110702 13028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:18.111567 13017 server_base.cc:1034] running on GCE node
I20260809 13:09:18.112154 13017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:18.113013 13017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:18.114228 13017 hybrid_clock.cc:648] HybridClock initialized: now 1786280958114160 us; error 30 us; skew 500 ppm
I20260809 13:09:18.114500 13017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:18.117417 13017 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:18.118850 13017 webserver.cc:460] Webserver started at http://127.10.181.130:44087/ using document root <none> and password file <none>
I20260809 13:09:18.119526 13017 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:18.119624 13017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:18.119926 13017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:18.122787 13017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "320f6e68ee87430cb0988d45d3b099fb"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "3a6f7c2a1695d617112d7f272f1f2a8c"
server_key_iv: "cf62f8a9d70eaeb64668fe743ce25e9b"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.123611 13017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "320f6e68ee87430cb0988d45d3b099fb"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "3a6f7c2a1695d617112d7f272f1f2a8c"
server_key_iv: "cf62f8a9d70eaeb64668fe743ce25e9b"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.128664 13017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:09:18.132378 13042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:18.133700 13017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:18.133862 13017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "320f6e68ee87430cb0988d45d3b099fb"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "3a6f7c2a1695d617112d7f272f1f2a8c"
server_key_iv: "cf62f8a9d70eaeb64668fe743ce25e9b"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.134083 13017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:18.154014 13017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:18.158069 13017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:18.158319 13017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:18.160957 13017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:18.161070 13017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:18.161222 13017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:18.161345 13017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:18.183857 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.161332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:52513 (local address 127.10.181.190:46639)
0809 13:09:18.161532 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.161537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.162963 (+ 1426us) server_negotiation.cc:407] Connection header received
0809 13:09:18.165133 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.165140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.165236 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.165365 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:18.167489 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.168670 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.171379 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.172063 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.172202 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.176858 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:18.176864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:18.176875 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:18.176920 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:18.180136 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.181053 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.181057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.181059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.181172 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.181729 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.181734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.181737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.182067 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:18.182235 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.183260 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.183494 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:09:18.185562 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.162012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.130:52513)
0809 13:09:18.162790 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.162826 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.164731 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.165630 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.165645 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.166428 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:18.167268 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.167297 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.168846 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.168852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.171214 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.171227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.172334 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.172343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.172627 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.173346 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:18.173387 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:18.176521 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:18.180279 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.180289 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.180306 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.180945 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.181262 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.181267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.181271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.181598 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.182215 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:18.182224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:18.182765 (+ 541us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.183140 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.183742 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":339,"threads_started":1}
I20260809 13:09:18.205367 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.191804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:50865 (local address 127.10.181.190:46639)
0809 13:09:18.191970 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.191975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.192649 (+ 674us) server_negotiation.cc:407] Connection header received
0809 13:09:18.192790 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.192795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.192857 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.192932 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:18.194036 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.194793 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.196126 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.196666 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.196804 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.199145 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:18.199151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:18.199156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:18.199190 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:18.202279 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.203188 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.203194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.203198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.203360 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.203800 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.203805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.203808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.204208 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:18.204364 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.204908 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.205135 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:09:18.205710 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.191888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.130:50865)
0809 13:09:18.192548 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.192567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.192689 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.192971 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.192975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.193429 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:18.193822 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.193836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.194941 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.194944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.195992 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.196005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.196875 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.196884 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.197042 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.197688 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:18.197716 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:18.198926 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:18.202407 (+ 3481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.202413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.202418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.203050 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.203502 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.203505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.203508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.203635 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.204332 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:18.204337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:18.204588 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.204800 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.205169 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":391,"threads_started":1}
I20260809 13:09:18.212805 13017 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:36873
I20260809 13:09:18.212845 13219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:36873 every 8 connection(s)
I20260809 13:09:18.213984 13017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:18.215013 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13017
I20260809 13:09:18.215217 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:18.215739 10966 external_mini_cluster.cc:1428] Setting key 104556003cbffc3d3b07550d053500a6
I20260809 13:09:18.216034 13221 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46639
I20260809 13:09:18.216245 13221 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:18.216977 13221 heartbeater.cc:511] Master 127.10.181.190:46639 requested a full tablet report, sending...
I20260809 13:09:18.218415 12698 ts_manager.cc:194] Registered new tserver with Master: 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873)
I20260809 13:09:18.219290 12698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:52513
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:18.257555 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46639
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:18.578922 13227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:18.579191 13227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:18.579291 13227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:18.586256 13227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:18.586351 13227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:18.586439 13227 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:18.586517 13227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:18.595928 13227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:46639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:18.597146 13227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:18.598480 13227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:18.611275 13234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:18.611341 13237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:18.611246 13235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:18.611581 13227 server_base.cc:1034] running on GCE node
I20260809 13:09:18.612381 13227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:18.613284 13227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:18.614516 13227 hybrid_clock.cc:648] HybridClock initialized: now 1786280958614429 us; error 46 us; skew 500 ppm
I20260809 13:09:18.614804 13227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:18.617843 13227 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:18.619213 13227 webserver.cc:460] Webserver started at http://127.10.181.131:40997/ using document root <none> and password file <none>
I20260809 13:09:18.619890 13227 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:18.619992 13227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:18.620319 13227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:18.623317 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "3b9b5a0f28094ed18f16a46409bf6e2d"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "5dd675242fee0bf300970133987e2060"
server_key_iv: "6df6e287850e758e2ca4ae33a7262568"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.624090 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "3b9b5a0f28094ed18f16a46409bf6e2d"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "5dd675242fee0bf300970133987e2060"
server_key_iv: "6df6e287850e758e2ca4ae33a7262568"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.629176 13227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 13:09:18.632525 13248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:18.633797 13227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:18.633950 13227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "3b9b5a0f28094ed18f16a46409bf6e2d"
format_stamp: "Formatted at 2026-08-09 13:09:18 on dist-test-slave-42z9"
server_key: "5dd675242fee0bf300970133987e2060"
server_key_iv: "6df6e287850e758e2ca4ae33a7262568"
server_key_version: "encryptionkey@0"
I20260809 13:09:18.634145 13227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:18.645691 13227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:18.649257 13227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:18.649583 13227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:18.651999 13227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:18.652078 13227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:18.652199 13227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:18.652259 13227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:18.676251 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.652733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:52053 (local address 127.10.181.190:46639)
0809 13:09:18.652914 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.652921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.654248 (+ 1327us) server_negotiation.cc:407] Connection header received
0809 13:09:18.656561 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.656567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.656692 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.656804 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:18.659227 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.660189 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.663100 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.663633 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.663776 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.669067 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:18.669075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:18.669080 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:18.669127 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:18.672489 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.673380 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.673386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.673390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.673574 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.674142 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.674147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.674151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.674424 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:18.674566 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.675640 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.675872 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:09:18.677572 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.653184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.131:52053)
0809 13:09:18.654028 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.654076 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.656184 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.657119 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.657135 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.658010 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:18.658997 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.659030 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.660389 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.660396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.662930 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.662951 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.663910 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.663918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.664249 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.665055 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:18.665104 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:18.668745 (+ 3641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:18.672626 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.672635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.672651 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.673251 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.673730 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.673735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.673738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.674016 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.674670 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:18.674679 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:18.675121 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.675473 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.676048 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":356,"threads_started":1}
I20260809 13:09:18.687469 13227 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:33039
I20260809 13:09:18.687490 13436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:33039 every 8 connection(s)
I20260809 13:09:18.688740 13227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:18.690838 13437 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46639
I20260809 13:09:18.691113 13437 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:18.692070 13437 heartbeater.cc:511] Master 127.10.181.190:46639 requested a full tablet report, sending...
I20260809 13:09:18.693929 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13227
I20260809 13:09:18.694056 12698 ts_manager.cc:194] Registered new tserver with Master: 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:18.694131 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:18.694630 10966 external_mini_cluster.cc:1428] Setting key 77fc5f0e05c421d92abd2b19b2540a4a
I20260809 13:09:18.695278 12698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:52053
I20260809 13:09:18.699782 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:18.703084 13011 heartbeater.cc:503] Master 127.10.181.190:46639 was elected leader, sending a full tablet report...
I20260809 13:09:18.710263 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.698285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:38305 (local address 127.10.181.190:46639)
0809 13:09:18.698455 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.698459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.699015 (+ 556us) server_negotiation.cc:407] Connection header received
0809 13:09:18.699161 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.699166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.699235 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.699310 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:18.700254 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.701037 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.702293 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.702695 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.702801 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.704967 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:18.704979 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:18.704983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:18.705013 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:18.707675 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.708444 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.708448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.708451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.708594 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.708954 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.708957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.708959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.709213 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:18.709335 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.709793 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.709965 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:09:18.710377 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.698369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.10.181.131:38305)
0809 13:09:18.698905 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.698921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.699053 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.699358 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.699363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.699835 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:18.700106 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.700117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.701162 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.701168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.702158 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.702172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.702863 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.702869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.703025 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.703832 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:18.703853 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:18.704789 (+ 936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:18.707869 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.707875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.707879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.708315 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.708699 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:18.708704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:18.708708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:18.708859 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:18.709324 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:18.709328 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:18.709553 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.709712 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.709978 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":291,"threads_started":1}
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 13:09:18 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:18.751520 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.738245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.10.181.190:46639)
0809 13:09:18.738536 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.738542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.738740 (+ 198us) server_negotiation.cc:407] Connection header received
0809 13:09:18.738935 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.738941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.739011 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.739096 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:18.740460 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.741205 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.742506 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.742940 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.743051 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.746710 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:18.746715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:18.746718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:18.746751 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:18.749008 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.749637 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.749643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.749647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.749785 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:18.750140 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:18.750144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:18.750147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:18.750410 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:18.750651 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.750951 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.751174 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260809 13:09:18.755548 12697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:18.762429 12697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:18.784547 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.774383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:34751 (local address 127.0.0.1:45450)
0809 13:09:18.775101 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.775144 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.775300 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.776809 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.776817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.776848 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:18.777534 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.777550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.779676 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.779682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.782102 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.782116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.783665 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.783671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.783852 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.783891 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.784002 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.784190 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
I20260809 13:09:18.784526 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.774864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.10.181.129:34751)
0809 13:09:18.775484 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.775494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.775525 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:09:18.775655 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.775663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.776554 (+ 891us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.777779 (+ 1225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:18.777977 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.779520 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.782290 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.783475 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.783641 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.783819 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.784069 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.784227 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":326,"threads_started":1}
I20260809 13:09:18.785084 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.774386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:36873 (local address 127.0.0.1:38800)
0809 13:09:18.775841 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.775858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.775973 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.777221 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.777226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.777251 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:18.777725 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.777738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.779629 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.779635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.782391 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.782406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.784241 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.784248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.784422 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.784453 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.784542 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.784678 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":160,"thread_start_us":1051,"threads_started":1}
I20260809 13:09:18.785809 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.774433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.10.181.130:36873)
0809 13:09:18.776435 (+ 2002us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.776444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.776469 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:09:18.776569 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.776577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.776802 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.777007 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:18.777914 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.779465 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.782566 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.783784 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.783938 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.784098 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.785311 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.785506 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1840,"thread_start_us":5381,"threads_started":1}
I20260809 13:09:18.789584 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.776439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33039 (local address 127.0.0.1:51706)
0809 13:09:18.781634 (+ 5195us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.781650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.781768 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.782413 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.782419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.782445 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:18.782819 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.782831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.784719 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.784724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.787381 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.787394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.788842 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.788848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.789020 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.789052 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.789136 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.789255 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5070,"thread_start_us":4886,"threads_started":1}
I20260809 13:09:18.789216 13119 tablet_service.cc:1467] Processing CreateTablet for tablet 62de8617952b41aab02b780fda50863e (DEFAULT_TABLE table=test-table [id=65b37f2d771343f8888f6b9ba0590970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.789216 12906 tablet_service.cc:1467] Processing CreateTablet for tablet 62de8617952b41aab02b780fda50863e (DEFAULT_TABLE table=test-table [id=65b37f2d771343f8888f6b9ba0590970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.790482 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.777071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.10.181.131:33039)
0809 13:09:18.781173 (+ 4102us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.781180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.781879 (+ 699us) server_negotiation.cc:407] Connection header received
0809 13:09:18.781980 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.781988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.782209 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.782407 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:18.783021 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.784570 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.787532 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.788663 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.788842 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.789006 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.789187 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.789343 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3922,"thread_start_us":3759,"threads_started":1}
I20260809 13:09:18.790782 13119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de8617952b41aab02b780fda50863e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.790900 12906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de8617952b41aab02b780fda50863e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.793150 13327 tablet_service.cc:1467] Processing CreateTablet for tablet 62de8617952b41aab02b780fda50863e (DEFAULT_TABLE table=test-table [id=65b37f2d771343f8888f6b9ba0590970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.794328 13327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62de8617952b41aab02b780fda50863e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.805169 13475 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Bootstrap starting.
I20260809 13:09:18.806020 13480 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Bootstrap starting.
I20260809 13:09:18.807485 13481 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Bootstrap starting.
I20260809 13:09:18.809340 13475 tablet_bootstrap.cc:654] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:18.810392 13481 tablet_bootstrap.cc:654] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:18.811331 13481 log.cc:824] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:18.811681 13475 log.cc:824] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:18.812441 13480 tablet_bootstrap.cc:654] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:18.813675 13480 log.cc:824] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:18.814285 13475 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: No bootstrap required, opened a new log
I20260809 13:09:18.814313 13481 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: No bootstrap required, opened a new log
I20260809 13:09:18.814546 13481 ts_tablet_manager.cc:1397] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 13:09:18.814569 13475 ts_tablet_manager.cc:1397] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 13:09:18.816524 13480 tablet_bootstrap.cc:492] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: No bootstrap required, opened a new log
I20260809 13:09:18.816849 13480 ts_tablet_manager.cc:1397] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260809 13:09:18.821902 13475 raft_consensus.cc:348] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.822233 13475 raft_consensus.cc:374] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:18.822331 13475 raft_consensus.cc:729] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320f6e68ee87430cb0988d45d3b099fb, State: Initialized, Role: FOLLOWER
I20260809 13:09:18.822888 13475 consensus_queue.cc:260] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.823930 13221 heartbeater.cc:503] Master 127.10.181.190:46639 was elected leader, sending a full tablet report...
I20260809 13:09:18.824652 13481 raft_consensus.cc:348] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.825116 13481 raft_consensus.cc:374] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:18.825255 13481 raft_consensus.cc:729] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9b5a0f28094ed18f16a46409bf6e2d, State: Initialized, Role: FOLLOWER
I20260809 13:09:18.825817 13475 ts_tablet_manager.cc:1428] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 13:09:18.825953 13481 consensus_queue.cc:260] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.827023 13437 heartbeater.cc:503] Master 127.10.181.190:46639 was elected leader, sending a full tablet report...
I20260809 13:09:18.827085 13480 raft_consensus.cc:348] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.827486 13480 raft_consensus.cc:374] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:18.827595 13480 raft_consensus.cc:729] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e93237324174f569cfc9b4c9654bfdc, State: Initialized, Role: FOLLOWER
I20260809 13:09:18.827677 13481 ts_tablet_manager.cc:1428] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 13:09:18.828208 13480 consensus_queue.cc:260] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.834549 13480 ts_tablet_manager.cc:1428] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
I20260809 13:09:18.873320 13488 raft_consensus.cc:482] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:18.873716 13488 raft_consensus.cc:504] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.875535 13488 leader_election.cc:288] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873), 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751)
I20260809 13:09:18.881842 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.875983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:36873 (local address 127.10.181.131:42887)
0809 13:09:18.876280 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.876301 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.876443 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.876909 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.876914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.876946 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:18.877229 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.877247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.878584 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.878589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.879987 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.880000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.881086 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.881092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.881270 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.881295 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.881377 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.881533 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":19}
I20260809 13:09:18.881991 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.875908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:42887 (local address 127.10.181.130:36873)
0809 13:09:18.876131 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.876137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.876374 (+ 237us) server_negotiation.cc:407] Connection header received
0809 13:09:18.876674 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.876679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.876759 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.876852 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:18.877405 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.878470 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.880154 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.880938 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.881087 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.881175 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.881422 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.881591 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:09:18.882650 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.875908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60917 (local address 127.10.181.129:34751)
0809 13:09:18.876139 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.876144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.877574 (+ 1430us) server_negotiation.cc:407] Connection header received
0809 13:09:18.877753 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.877756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.877819 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.877887 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:18.878398 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.879099 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.880871 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.881757 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.881942 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.882017 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.882333 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.882449 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:09:18.883212 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.876731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:34751 (local address 127.10.181.131:60917)
0809 13:09:18.877469 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:18.877488 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:18.877628 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:18.877923 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:18.877928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:18.877954 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:18.878274 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.878285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.879234 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.879238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.880719 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:18.880731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.881961 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.881967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:18.882137 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.882172 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:09:18.882824 (+ 652us) client_negotiation.cc:240] Negotiation successful
0809 13:09:18.882945 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":2,"thread_start_us":395,"threads_started":1}
I20260809 13:09:18.883162 13146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de8617952b41aab02b780fda50863e" candidate_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320f6e68ee87430cb0988d45d3b099fb" is_pre_election: true
I20260809 13:09:18.883744 13146 raft_consensus.cc:2454] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b9b5a0f28094ed18f16a46409bf6e2d in term 0.
I20260809 13:09:18.884212 12934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de8617952b41aab02b780fda50863e" candidate_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e93237324174f569cfc9b4c9654bfdc" is_pre_election: true
I20260809 13:09:18.884409 13251 leader_election.cc:302] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 3b9b5a0f28094ed18f16a46409bf6e2d; no voters:
I20260809 13:09:18.884697 12934 raft_consensus.cc:2454] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b9b5a0f28094ed18f16a46409bf6e2d in term 0.
I20260809 13:09:18.884791 13488 raft_consensus.cc:2781] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:18.884949 13488 raft_consensus.cc:482] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:18.885054 13488 raft_consensus.cc:3037] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:18.887538 13488 raft_consensus.cc:504] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.888119 13488 leader_election.cc:288] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [CANDIDATE]: Term 1 election: Requested vote from peers 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873), 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751)
I20260809 13:09:18.888468 13146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de8617952b41aab02b780fda50863e" candidate_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320f6e68ee87430cb0988d45d3b099fb"
I20260809 13:09:18.888602 12934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62de8617952b41aab02b780fda50863e" candidate_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e93237324174f569cfc9b4c9654bfdc"
I20260809 13:09:18.888666 13146 raft_consensus.cc:3037] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:18.888798 12934 raft_consensus.cc:3037] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:18.890895 13146 raft_consensus.cc:2454] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b9b5a0f28094ed18f16a46409bf6e2d in term 1.
I20260809 13:09:18.891047 12934 raft_consensus.cc:2454] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b9b5a0f28094ed18f16a46409bf6e2d in term 1.
I20260809 13:09:18.891264 13251 leader_election.cc:302] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 3b9b5a0f28094ed18f16a46409bf6e2d; no voters:
I20260809 13:09:18.891535 13488 raft_consensus.cc:2781] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:18.891957 13488 raft_consensus.cc:686] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 LEADER]: Becoming Leader. State: Replica: 3b9b5a0f28094ed18f16a46409bf6e2d, State: Running, Role: LEADER
I20260809 13:09:18.892405 13488 consensus_queue.cc:237] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:18.897149 12697 catalog_manager.cc:5495] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131). New cstate: current_term: 1 leader_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } health_report { overall_health: HEALTHY } } }
I20260809 13:09:18.917145 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:18.912237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.10.181.131:33039)
0809 13:09:18.912451 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:18.912457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:18.912547 (+ 90us) server_negotiation.cc:407] Connection header received
0809 13:09:18.912809 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:18.912813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:18.912893 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:18.913005 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:18.913290 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.914002 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.915273 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:18.915673 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:18.915757 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:18.916175 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:18.916646 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:18.916746 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:18.916829 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 13:09:18.916890 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:09:18.924942 13146 raft_consensus.cc:1261] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Refusing update from remote peer 3b9b5a0f28094ed18f16a46409bf6e2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:18.924944 12934 raft_consensus.cc:1261] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Refusing update from remote peer 3b9b5a0f28094ed18f16a46409bf6e2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:18.925853 13491 consensus_queue.cc:1035] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:18.926177 13488 consensus_queue.cc:1035] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:18.936646 13496 mvcc.cc:205] Tried to move back new op lower bound from 7316606807747252224 to 7316606807627116544. Current Snapshot: MvccSnapshot[applied={T|T < 7316606807747252224}]
I20260809 13:09:18.940161 13497 mvcc.cc:205] Tried to move back new op lower bound from 7316606807747252224 to 7316606807627116544. Current Snapshot: MvccSnapshot[applied={T|T < 7316606807747252224}]
I20260809 13:09:18.958276 12697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:18.958516 12697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:18.962010 12697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Sending DeleteTablet for 3 replicas of tablet 62de8617952b41aab02b780fda50863e
I20260809 13:09:18.963172 12906 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de8617952b41aab02b780fda50863e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:18 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:45450
I20260809 13:09:18.963171 13119 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de8617952b41aab02b780fda50863e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:18 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:38800
I20260809 13:09:18.963199 13327 tablet_service.cc:1514] Processing DeleteTablet for tablet 62de8617952b41aab02b780fda50863e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:18 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:51706
I20260809 13:09:18.970630 12697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:18.971647 12697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:18.975641 13511 tablet_replica.cc:330] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: stopping tablet replica
I20260809 13:09:18.976126 13511 raft_consensus.cc:2229] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:18.976670 13511 raft_consensus.cc:2258] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:18.977660 13509 tablet_replica.cc:330] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: stopping tablet replica
I20260809 13:09:18.978133 13509 raft_consensus.cc:2229] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:18.979094 13509 raft_consensus.cc:2258] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:18.980005 13511 ts_tablet_manager.cc:1905] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:18.980163 13510 tablet_replica.cc:330] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: stopping tablet replica
I20260809 13:09:18.981689 13509 ts_tablet_manager.cc:1905] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:18.983481 12906 tablet_service.cc:1467] Processing CreateTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d (DEFAULT_TABLE table=test-table [id=7982ed47c02a4c1ba05402fffc2ed997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.983947 12906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64f5583be6645c1a4bd1ff44cd1899d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.986280 13510 raft_consensus.cc:2229] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:18.986891 13327 tablet_service.cc:1467] Processing CreateTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d (DEFAULT_TABLE table=test-table [id=7982ed47c02a4c1ba05402fffc2ed997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.987367 13327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64f5583be6645c1a4bd1ff44cd1899d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.987967 13119 tablet_service.cc:1467] Processing CreateTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d (DEFAULT_TABLE table=test-table [id=7982ed47c02a4c1ba05402fffc2ed997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:18.988402 13119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64f5583be6645c1a4bd1ff44cd1899d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:18.989342 13509 ts_tablet_manager.cc:1918] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:18.989641 13509 log.cc:1196] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/62de8617952b41aab02b780fda50863e
I20260809 13:09:18.990739 13509 ts_tablet_manager.cc:1939] T 62de8617952b41aab02b780fda50863e P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting consensus metadata
I20260809 13:09:18.992767 12672 catalog_manager.cc:4841] TS 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039): tablet 62de8617952b41aab02b780fda50863e (table test-table [id=65b37f2d771343f8888f6b9ba0590970]) successfully deleted
I20260809 13:09:18.993906 13510 raft_consensus.cc:2258] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:18.994330 13481 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Bootstrap starting.
I20260809 13:09:18.995998 13481 tablet_bootstrap.cc:654] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:18.996724 13511 ts_tablet_manager.cc:1918] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:18.996896 13511 log.cc:1196] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/62de8617952b41aab02b780fda50863e
I20260809 13:09:18.997552 13511 ts_tablet_manager.cc:1939] T 62de8617952b41aab02b780fda50863e P 5e93237324174f569cfc9b4c9654bfdc: Deleting consensus metadata
I20260809 13:09:18.999333 13480 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Bootstrap starting.
I20260809 13:09:18.999622 12674 catalog_manager.cc:4841] TS 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751): tablet 62de8617952b41aab02b780fda50863e (table test-table [id=65b37f2d771343f8888f6b9ba0590970]) successfully deleted
I20260809 13:09:19.001021 13480 tablet_bootstrap.cc:654] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:19.001699 13510 ts_tablet_manager.cc:1905] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:19.004896 13475 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Bootstrap starting.
I20260809 13:09:19.006647 13475 tablet_bootstrap.cc:654] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:19.006992 13481 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: No bootstrap required, opened a new log
I20260809 13:09:19.007913 13480 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: No bootstrap required, opened a new log
I20260809 13:09:19.008121 13480 ts_tablet_manager.cc:1397] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 13:09:19.008239 13481 ts_tablet_manager.cc:1397] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260809 13:09:19.008919 13510 ts_tablet_manager.cc:1918] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:19.009153 13510 log.cc:1196] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/62de8617952b41aab02b780fda50863e
I20260809 13:09:19.009024 13480 raft_consensus.cc:348] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.009037 13481 raft_consensus.cc:348] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.009243 13480 raft_consensus.cc:374] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.009243 13481 raft_consensus.cc:374] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.009339 13481 raft_consensus.cc:729] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9b5a0f28094ed18f16a46409bf6e2d, State: Initialized, Role: FOLLOWER
I20260809 13:09:19.009488 13480 raft_consensus.cc:729] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e93237324174f569cfc9b4c9654bfdc, State: Initialized, Role: FOLLOWER
I20260809 13:09:19.009671 13481 consensus_queue.cc:260] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.009984 13510 ts_tablet_manager.cc:1939] T 62de8617952b41aab02b780fda50863e P 320f6e68ee87430cb0988d45d3b099fb: Deleting consensus metadata
I20260809 13:09:19.009843 13480 consensus_queue.cc:260] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.010265 13481 ts_tablet_manager.cc:1428] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:09:19.014633 12670 catalog_manager.cc:4841] TS 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873): tablet 62de8617952b41aab02b780fda50863e (table test-table [id=65b37f2d771343f8888f6b9ba0590970]) successfully deleted
I20260809 13:09:19.015555 13480 ts_tablet_manager.cc:1428] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260809 13:09:19.016502 13475 tablet_bootstrap.cc:492] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: No bootstrap required, opened a new log
I20260809 13:09:19.016664 13475 ts_tablet_manager.cc:1397] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260809 13:09:19.017489 13475 raft_consensus.cc:348] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.017728 13475 raft_consensus.cc:374] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.017823 13475 raft_consensus.cc:729] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320f6e68ee87430cb0988d45d3b099fb, State: Initialized, Role: FOLLOWER
I20260809 13:09:19.018069 13475 consensus_queue.cc:260] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.018769 13475 ts_tablet_manager.cc:1428] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 13:09:19.019351 13487 raft_consensus.cc:482] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:19.019546 13487 raft_consensus.cc:504] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.021203 13489 raft_consensus.cc:482] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:19.021243 13487 leader_election.cc:288] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751), 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:19.021379 13489 raft_consensus.cc:504] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.023092 13489 leader_election.cc:288] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873), 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:19.059155 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.021811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:53681 (local address 127.10.181.129:34751)
0809 13:09:19.022012 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.022019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.049757 (+ 27738us) server_negotiation.cc:407] Connection header received
0809 13:09:19.052242 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.052247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.052342 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.052399 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:19.053218 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.054321 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.056780 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.058000 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.058123 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.058188 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.058722 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.058851 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:09:19.061302 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.021582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:34751 (local address 127.10.181.130:53681)
0809 13:09:19.049665 (+ 28083us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.049684 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.052077 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.052573 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.052577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.052603 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:19.053042 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.053054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.054491 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.054495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.056614 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.056630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.058373 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.058379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.058543 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.058575 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.059619 (+ 1044us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.060887 (+ 1268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27938,"thread_start_us":30650,"threads_started":1}
I20260809 13:09:19.062259 12934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "320f6e68ee87430cb0988d45d3b099fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e93237324174f569cfc9b4c9654bfdc" is_pre_election: true
I20260809 13:09:19.062413 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.024080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:60349 (local address 127.10.181.131:33039)
0809 13:09:19.054346 (+ 30266us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.054351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.054374 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:09:19.054425 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.054430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.054504 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.054607 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:19.055598 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.056854 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.059221 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.060589 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.060777 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.060850 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.061875 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.062019 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30121,"thread_start_us":27884,"threads_started":1}
I20260809 13:09:19.062541 12934 raft_consensus.cc:2454] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320f6e68ee87430cb0988d45d3b099fb in term 0.
I20260809 13:09:19.062844 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.023662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33039 (local address 127.10.181.130:60349)
0809 13:09:19.052532 (+ 28870us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.052547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.052640 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.054901 (+ 2261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.054907 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.054937 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:19.055348 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.055361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.057015 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.057021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.059047 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.059064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.061008 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.061015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.061228 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.061266 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.061356 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.061507 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28758,"thread_start_us":25656,"threads_started":1}
I20260809 13:09:19.063272 13048 leader_election.cc:302] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 5e93237324174f569cfc9b4c9654bfdc; no voters:
I20260809 13:09:19.063874 13487 raft_consensus.cc:2781] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:19.063998 13487 raft_consensus.cc:482] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:19.064091 13487 raft_consensus.cc:3037] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.063875 13367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "320f6e68ee87430cb0988d45d3b099fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" is_pre_election: true
I20260809 13:09:19.064324 13367 raft_consensus.cc:2454] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320f6e68ee87430cb0988d45d3b099fb in term 0.
I20260809 13:09:19.067430 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.023956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:45801 (local address 127.10.181.130:36873)
0809 13:09:19.024122 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.024128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.045405 (+ 21277us) server_negotiation.cc:407] Connection header received
0809 13:09:19.045748 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.045755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.045838 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.045911 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:19.047053 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.048326 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.051975 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.053335 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.053529 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.053607 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.067025 (+ 13418us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.067166 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:09:19.067355 13487 raft_consensus.cc:504] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.068147 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.023290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:36873 (local address 127.10.181.129:45801)
0809 13:09:19.045271 (+ 21981us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.045298 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.045567 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.046149 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.046154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.046185 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:19.046617 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.046630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.050078 (+ 3448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.050082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.051816 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.051829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.053774 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.053784 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.053950 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.053985 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.067638 (+ 13653us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.067781 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21807,"thread_start_us":25306,"threads_started":1}
I20260809 13:09:19.068570 13367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "320f6e68ee87430cb0988d45d3b099fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d"
I20260809 13:09:19.068779 13367 raft_consensus.cc:3037] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.068727 13146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320f6e68ee87430cb0988d45d3b099fb" is_pre_election: true
I20260809 13:09:19.068991 13146 raft_consensus.cc:2379] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e93237324174f569cfc9b4c9654bfdc in current term 1: Already voted for candidate 320f6e68ee87430cb0988d45d3b099fb in this term.
I20260809 13:09:19.071389 13367 raft_consensus.cc:2454] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320f6e68ee87430cb0988d45d3b099fb in term 1.
I20260809 13:09:19.071666 13487 leader_election.cc:288] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [CANDIDATE]: Term 1 election: Requested vote from peers 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751), 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:19.071870 13047 leader_election.cc:302] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 3b9b5a0f28094ed18f16a46409bf6e2d; no voters:
I20260809 13:09:19.072333 13487 raft_consensus.cc:2781] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:19.072361 12934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "320f6e68ee87430cb0988d45d3b099fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e93237324174f569cfc9b4c9654bfdc"
I20260809 13:09:19.072572 12934 raft_consensus.cc:3037] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.075682 12934 raft_consensus.cc:2454] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320f6e68ee87430cb0988d45d3b099fb in term 1.
I20260809 13:09:19.080849 13487 raft_consensus.cc:686] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 LEADER]: Becoming Leader. State: Replica: 320f6e68ee87430cb0988d45d3b099fb, State: Running, Role: LEADER
I20260809 13:09:19.081339 13487 consensus_queue.cc:237] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.082512 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.023670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33039 (local address 127.10.181.129:59133)
0809 13:09:19.048952 (+ 25282us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.048967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.049080 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.049656 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.049660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.049687 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:19.050083 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.050094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.054113 (+ 4019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.054117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.066801 (+ 12684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.066819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.081609 (+ 14790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.081617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.081831 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.081871 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.081971 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.082136 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25160,"thread_start_us":23100,"threads_started":1}
I20260809 13:09:19.082512 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.023347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:59133 (local address 127.10.181.131:33039)
0809 13:09:19.023513 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.023519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.049071 (+ 25552us) server_negotiation.cc:407] Connection header received
0809 13:09:19.049195 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.049200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.049316 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.049468 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:19.050291 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.053923 (+ 3632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.066968 (+ 13045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.068272 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.068418 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.068495 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.082057 (+ 13562us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.082217 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:09:19.083550 13367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64f5583be6645c1a4bd1ff44cd1899d" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" is_pre_election: true
I20260809 13:09:19.083827 13367 raft_consensus.cc:2379] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e93237324174f569cfc9b4c9654bfdc in current term 1: Already voted for candidate 320f6e68ee87430cb0988d45d3b099fb in this term.
I20260809 13:09:19.084475 12827 leader_election.cc:302] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e93237324174f569cfc9b4c9654bfdc; no voters: 320f6e68ee87430cb0988d45d3b099fb, 3b9b5a0f28094ed18f16a46409bf6e2d
I20260809 13:09:19.084939 13489 raft_consensus.cc:2726] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 13:09:19.090157 12697 catalog_manager.cc:5495] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130). New cstate: current_term: 1 leader_uuid: "320f6e68ee87430cb0988d45d3b099fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } health_report { overall_health: UNKNOWN } } }
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280956, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:19.147382 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.132129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46639 (local address 127.0.0.1:32958)
0809 13:09:19.132813 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.132843 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.132976 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.133338 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.133341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.134123 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:19.134402 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.134411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.135841 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.135845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.136856 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.136868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.137935 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.137941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.138080 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.138877 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:19.138895 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:19.141090 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:19.144862 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:19.144873 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:19.144879 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:19.145270 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:19.145724 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:19.145728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:19.145731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:19.145881 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:19.146549 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:19.146555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:19.146722 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.146858 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.147059 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":369,"threads_started":1}
I20260809 13:09:19.147629 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.132092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.10.181.190:46639)
0809 13:09:19.132290 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.132297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.132938 (+ 641us) server_negotiation.cc:407] Connection header received
0809 13:09:19.133089 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.133095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.133192 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.133306 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.134554 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.135722 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.136988 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.137716 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.137858 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.141281 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.141288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.141294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.141332 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.144709 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.145383 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.145389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.145393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.145607 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.145987 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.145992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.145996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.146413 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.146564 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.146992 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.147218 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:09:19.153497 12697 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:32958:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 13:09:19.154915 12697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:19.162460 12906 tablet_service.cc:1467] Processing CreateTablet for tablet 66046b17d5d2448ea28f9486106ac6ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ac5965cf4234e36896122d29f5d949b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:19.162747 13327 tablet_service.cc:1467] Processing CreateTablet for tablet 66046b17d5d2448ea28f9486106ac6ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ac5965cf4234e36896122d29f5d949b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:19.163136 12906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66046b17d5d2448ea28f9486106ac6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:19.163329 13327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66046b17d5d2448ea28f9486106ac6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:19.166028 13119 tablet_service.cc:1467] Processing CreateTablet for tablet 66046b17d5d2448ea28f9486106ac6ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ac5965cf4234e36896122d29f5d949b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:19.166570 13119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66046b17d5d2448ea28f9486106ac6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:19.170518 13480 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: Bootstrap starting.
I20260809 13:09:19.170622 13481 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: Bootstrap starting.
I20260809 13:09:19.172271 13481 tablet_bootstrap.cc:654] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:19.172317 13480 tablet_bootstrap.cc:654] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:19.174692 13475 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: Bootstrap starting.
I20260809 13:09:19.176462 13475 tablet_bootstrap.cc:654] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:19.181021 13480 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: No bootstrap required, opened a new log
I20260809 13:09:19.181200 13480 ts_tablet_manager.cc:1397] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260809 13:09:19.182045 13480 raft_consensus.cc:348] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.182298 13480 raft_consensus.cc:374] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.182369 13480 raft_consensus.cc:729] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e93237324174f569cfc9b4c9654bfdc, State: Initialized, Role: FOLLOWER
I20260809 13:09:19.182636 13480 consensus_queue.cc:260] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.183080 13489 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.183357 13489 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.183984 13481 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: No bootstrap required, opened a new log
I20260809 13:09:19.184227 13481 ts_tablet_manager.cc:1397] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260809 13:09:19.185068 13481 raft_consensus.cc:348] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.185393 13481 raft_consensus.cc:374] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.185555 13481 raft_consensus.cc:729] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9b5a0f28094ed18f16a46409bf6e2d, State: Initialized, Role: FOLLOWER
I20260809 13:09:19.185927 13481 consensus_queue.cc:260] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.186089 13480 ts_tablet_manager.cc:1428] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 13:09:19.186439 13503 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.186785 13503 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.188395 13481 ts_tablet_manager.cc:1428] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 13:09:19.189954 13475 tablet_bootstrap.cc:492] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: No bootstrap required, opened a new log
I20260809 13:09:19.190163 13475 ts_tablet_manager.cc:1397] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.002s
I20260809 13:09:19.191097 13475 raft_consensus.cc:348] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.191375 13475 raft_consensus.cc:374] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:19.191469 13475 raft_consensus.cc:729] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320f6e68ee87430cb0988d45d3b099fb, State: Initialized, Role: FOLLOWER
W20260809 13:09:19.191686 13441 tablet.cc:2369] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:09:19.192133 13475 consensus_queue.cc:260] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.192699 13475 ts_tablet_manager.cc:1428] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:09:19.192551 13487 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.192893 13487 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 13:09:19.197216 13013 tablet.cc:2369] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:09:19.217023 13223 tablet.cc:2369] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:09:19.254153 13489 raft_consensus.cc:482] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:19.254355 13489 raft_consensus.cc:504] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.255014 13489 leader_election.cc:288] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873), 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:19.255530 13146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66046b17d5d2448ea28f9486106ac6ce" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320f6e68ee87430cb0988d45d3b099fb" is_pre_election: true
I20260809 13:09:19.255602 13367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66046b17d5d2448ea28f9486106ac6ce" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" is_pre_election: true
I20260809 13:09:19.255808 13146 raft_consensus.cc:2454] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e93237324174f569cfc9b4c9654bfdc in term 0.
I20260809 13:09:19.255868 13367 raft_consensus.cc:2454] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e93237324174f569cfc9b4c9654bfdc in term 0.
I20260809 13:09:19.256330 12825 leader_election.cc:302] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 3b9b5a0f28094ed18f16a46409bf6e2d, 5e93237324174f569cfc9b4c9654bfdc; no voters:
I20260809 13:09:19.256649 13489 raft_consensus.cc:2781] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:19.256776 13489 raft_consensus.cc:482] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:19.256873 13489 raft_consensus.cc:3037] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.259131 13489 raft_consensus.cc:504] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.259735 13489 leader_election.cc:288] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 election: Requested vote from peers 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873), 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039)
I20260809 13:09:19.260134 13146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66046b17d5d2448ea28f9486106ac6ce" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320f6e68ee87430cb0988d45d3b099fb"
I20260809 13:09:19.260272 13367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66046b17d5d2448ea28f9486106ac6ce" candidate_uuid: "5e93237324174f569cfc9b4c9654bfdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d"
I20260809 13:09:19.260346 13146 raft_consensus.cc:3037] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.260497 13367 raft_consensus.cc:3037] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:19.263307 13146 raft_consensus.cc:2454] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e93237324174f569cfc9b4c9654bfdc in term 1.
I20260809 13:09:19.263487 13367 raft_consensus.cc:2454] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e93237324174f569cfc9b4c9654bfdc in term 1.
I20260809 13:09:19.263796 12825 leader_election.cc:302] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320f6e68ee87430cb0988d45d3b099fb, 5e93237324174f569cfc9b4c9654bfdc; no voters:
I20260809 13:09:19.264096 13489 raft_consensus.cc:2781] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:19.264590 13489 raft_consensus.cc:686] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [term 1 LEADER]: Becoming Leader. State: Replica: 5e93237324174f569cfc9b4c9654bfdc, State: Running, Role: LEADER
I20260809 13:09:19.265048 13489 consensus_queue.cc:237] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } }
I20260809 13:09:19.267969 13547 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: TxnStatusTablet state changed. Reason: New leader 5e93237324174f569cfc9b4c9654bfdc. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.268328 13547 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:19.269069 13549 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 13:09:19.269351 13549 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 13:09:19.269533 12698 catalog_manager.cc:5495] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129). New cstate: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } health_report { overall_health: UNKNOWN } } }
I20260809 13:09:19.291543 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.283592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:34751 (local address 127.0.0.1:45454)
0809 13:09:19.283814 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.283830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.283969 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.284640 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.284646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.284659 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:19.284859 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.284870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.286625 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.286631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.288356 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.288373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.289521 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.289527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.289742 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.289817 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.290944 (+ 1127us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.290967 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.291050 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.291207 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 13:09:19.292402 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.283561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.10.181.129:34751)
0809 13:09:19.283738 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.283745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.283901 (+ 156us) server_negotiation.cc:407] Connection header received
0809 13:09:19.284160 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.284165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.284292 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.284449 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:19.285053 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.286284 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.288538 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.289231 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.289354 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.290072 (+ 718us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.290726 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.290872 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.292016 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.292104 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:09:19.298003 13367 raft_consensus.cc:1261] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Refusing update from remote peer 5e93237324174f569cfc9b4c9654bfdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:19.298022 13146 raft_consensus.cc:1261] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Refusing update from remote peer 5e93237324174f569cfc9b4c9654bfdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:19.298781 13489 consensus_queue.cc:1035] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:19.299161 13547 consensus_queue.cc:1035] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:19.314795 13487 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: TxnStatusTablet state changed. Reason: New leader 5e93237324174f569cfc9b4c9654bfdc. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.315126 13487 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.316345 13503 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: TxnStatusTablet state changed. Reason: New leader 5e93237324174f569cfc9b4c9654bfdc. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.316655 13503 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.319198 13489 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.319533 13489 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:19.319736 13498 mvcc.cc:205] Tried to move back new op lower bound from 7316606809275232256 to 7316606809153449984. Current Snapshot: MvccSnapshot[applied={T|T < 7316606809275232256 or (T in {7316606809275232256})}]
I20260809 13:09:19.322621 13487 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.322894 13487 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 320f6e68ee87430cb0988d45d3b099fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.326187 13547 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.326434 13547 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 5e93237324174f569cfc9b4c9654bfdc: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:19.328668 13503 tablet_replica.cc:439] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e93237324174f569cfc9b4c9654bfdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 } } peers { permanent_uuid: "320f6e68ee87430cb0988d45d3b099fb" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36873 } } peers { permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 } } }
I20260809 13:09:19.328915 13503 tablet_replica.cc:442] T 66046b17d5d2448ea28f9486106ac6ce P 3b9b5a0f28094ed18f16a46409bf6e2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:19.337651 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.331796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.10.181.130:36873)
0809 13:09:19.332192 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.332198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.332217 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:09:19.332401 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.332406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.332498 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.332627 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:19.333027 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.333831 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.335194 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.335664 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.335783 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.336284 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.336884 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.337022 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.337203 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.337342 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260809 13:09:19.353610 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.348004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:49573 (local address 127.10.181.129:34751)
0809 13:09:19.348181 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.348191 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.348775 (+ 584us) server_negotiation.cc:407] Connection header received
0809 13:09:19.348967 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.348971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.349039 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.349124 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:19.349675 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.350399 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.351767 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.352290 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.352423 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.352821 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.352957 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.353033 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.353177 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.353258 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:09:19.353714 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.348032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:34751 (local address 127.10.181.130:49573)
0809 13:09:19.348674 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.348689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.348823 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.349144 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.349148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.349160 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:19.349518 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.349530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.350533 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.350538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.351628 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.351641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.352472 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.352479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.352605 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.352658 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.353055 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.353068 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.353138 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.353277 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":354,"threads_started":1}
I20260809 13:09:19.365087 13367 raft_consensus.cc:1261] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Refusing update from remote peer 320f6e68ee87430cb0988d45d3b099fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:19.365243 12934 raft_consensus.cc:1261] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Refusing update from remote peer 320f6e68ee87430cb0988d45d3b099fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:19.366003 13528 consensus_queue.cc:1035] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b9b5a0f28094ed18f16a46409bf6e2d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:19.366400 13487 consensus_queue.cc:1035] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e93237324174f569cfc9b4c9654bfdc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:19.440698 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.432174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:40119 (local address 127.10.181.130:36873)
0809 13:09:19.432367 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.432373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.434196 (+ 1823us) server_negotiation.cc:407] Connection header received
0809 13:09:19.434380 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.434386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.434454 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.434554 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:19.435291 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.436321 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.438633 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.439156 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.439269 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.439841 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.439997 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.440067 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.440311 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.440397 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:09:19.441704 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.431732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:36873 (local address 127.10.181.129:40119)
0809 13:09:19.434093 (+ 2361us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:19.434110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:19.434248 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:19.434694 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:19.434697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:19.434708 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:19.435088 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.435100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.436493 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.436497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.438467 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:19.438485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.439346 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.439354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:19.439538 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.439623 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:19.440156 (+ 533us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:19.440175 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 13:09:19.441148 (+ 973us) client_negotiation.cc:240] Negotiation successful
0809 13:09:19.441291 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2159,"thread_start_us":2002,"threads_started":1}
I20260809 13:09:19.496089 12697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:19.496397 12697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:19.500557 12697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a25b37de833421c87872e3b2bfe170e: Sending DeleteTablet for 3 replicas of tablet f64f5583be6645c1a4bd1ff44cd1899d
I20260809 13:09:19.502261 13327 tablet_service.cc:1514] Processing DeleteTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:51706
I20260809 13:09:19.502678 13119 tablet_service.cc:1514] Processing DeleteTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:38800
I20260809 13:09:19.502993 13510 tablet_replica.cc:330] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: stopping tablet replica
I20260809 13:09:19.503207 13510 raft_consensus.cc:2229] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:19.503724 13510 raft_consensus.cc:2258] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:19.505116 13510 ts_tablet_manager.cc:1905] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:19.506505 12906 tablet_service.cc:1514] Processing DeleteTablet for tablet f64f5583be6645c1a4bd1ff44cd1899d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:45450
I20260809 13:09:19.512708 13510 ts_tablet_manager.cc:1918] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:19.514058 13510 log.cc:1196] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/f64f5583be6645c1a4bd1ff44cd1899d
I20260809 13:09:19.514627 13510 ts_tablet_manager.cc:1939] T f64f5583be6645c1a4bd1ff44cd1899d P 320f6e68ee87430cb0988d45d3b099fb: Deleting consensus metadata
I20260809 13:09:19.518364 12670 catalog_manager.cc:4841] TS 320f6e68ee87430cb0988d45d3b099fb (127.10.181.130:36873): tablet f64f5583be6645c1a4bd1ff44cd1899d (table test-table [id=7982ed47c02a4c1ba05402fffc2ed997]) successfully deleted
I20260809 13:09:19.521157 13577 tablet_replica.cc:330] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: stopping tablet replica
I20260809 13:09:19.521377 13577 raft_consensus.cc:2229] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:19.521692 13577 raft_consensus.cc:2258] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:19.522197 13574 tablet_replica.cc:330] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: stopping tablet replica
I20260809 13:09:19.522394 13574 raft_consensus.cc:2229] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:19.522609 13574 raft_consensus.cc:2258] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:19.523845 13574 ts_tablet_manager.cc:1905] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:19.524775 13577 ts_tablet_manager.cc:1905] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:19.532472 13577 ts_tablet_manager.cc:1918] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:19.532624 13577 log.cc:1196] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/f64f5583be6645c1a4bd1ff44cd1899d
I20260809 13:09:19.533166 13577 ts_tablet_manager.cc:1939] T f64f5583be6645c1a4bd1ff44cd1899d P 5e93237324174f569cfc9b4c9654bfdc: Deleting consensus metadata
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:19.534363 12674 catalog_manager.cc:4841] TS 5e93237324174f569cfc9b4c9654bfdc (127.10.181.129:34751): tablet f64f5583be6645c1a4bd1ff44cd1899d (table test-table [id=7982ed47c02a4c1ba05402fffc2ed997]) successfully deleted
I20260809 13:09:19.541597 13574 ts_tablet_manager.cc:1918] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:19.541754 13574 log.cc:1196] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/f64f5583be6645c1a4bd1ff44cd1899d
I20260809 13:09:19.542210 13574 ts_tablet_manager.cc:1939] T f64f5583be6645c1a4bd1ff44cd1899d P 3b9b5a0f28094ed18f16a46409bf6e2d: Deleting consensus metadata
I20260809 13:09:19.543009 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.521777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.10.181.129:34751)
0809 13:09:19.521974 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.521981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.523393 (+ 1412us) server_negotiation.cc:407] Connection header received
0809 13:09:19.523688 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.523694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.523783 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.523860 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.526067 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.527262 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.529169 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.529937 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.530047 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.535304 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.535319 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.535339 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.535379 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.539432 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.540328 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.540338 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.540342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.540491 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.540974 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.540978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.540981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.541421 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.541791 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.542398 (+ 607us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.542623 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:09:19.543325 12672 catalog_manager.cc:4841] TS 3b9b5a0f28094ed18f16a46409bf6e2d (127.10.181.131:33039): tablet f64f5583be6645c1a4bd1ff44cd1899d (table test-table [id=7982ed47c02a4c1ba05402fffc2ed997]) successfully deleted
W20260809 13:09:19.544397 13004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45466
I20260809 13:09:19.565127 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.548060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.10.181.190:46639)
0809 13:09:19.548268 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.548274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.549978 (+ 1704us) server_negotiation.cc:407] Connection header received
0809 13:09:19.550155 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.550160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.550242 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.550342 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.552020 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.553099 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.554935 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.555611 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.555732 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.558760 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.558767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.558771 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.558812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.562186 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.562972 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.562978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.562981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.563128 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.563534 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.563539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.563542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.563897 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.564175 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.564497 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.564727 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 13:09:19.565723 12697 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32960
I20260809 13:09:19.568148 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12801
I20260809 13:09:19.585228 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.570542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.10.181.129:34751)
0809 13:09:19.570708 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.570714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.570731 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:09:19.570786 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.570791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.570862 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.570979 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.572566 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.573715 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.575447 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.576152 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.576280 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.579052 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.579059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.579063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.579102 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.582418 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.583181 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.583187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.583190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.583334 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.583710 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.583715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.583718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.584068 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.584312 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.584620 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.584854 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 13:09:19.585805 13004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45470
W20260809 13:09:19.586404 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12801: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:19.586599 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12801
I20260809 13:09:19.615331 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13017
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:19.631704 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.616071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.10.181.130:36873)
0809 13:09:19.616262 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.616268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.616323 (+ 55us) server_negotiation.cc:407] Connection header received
0809 13:09:19.616537 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.616541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.616610 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.616697 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.618000 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.618781 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.620026 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.620493 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.620606 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.624813 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.624830 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.624852 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.624900 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.628695 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.629289 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.629300 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.629304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.629501 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.629829 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.629835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.629838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.630314 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.630660 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.630890 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.631142 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 13:09:19.632475 13214 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38818
W20260809 13:09:19.633159 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13017: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:19.633342 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13017
I20260809 13:09:19.664590 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13227
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[12625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:19.678354 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:19.665321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.10.181.131:33039)
0809 13:09:19.665804 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:19.665809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:19.665827 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:09:19.665884 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:19.665888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:19.665954 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:19.666049 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:19.667210 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.667883 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.668973 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:19.669394 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:19.669575 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:19.673360 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:19.673376 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:19.673394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:19.673428 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:19.676100 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.676650 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.676657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.676660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.676788 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:19.677057 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:19.677060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:19.677062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:19.677408 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:19.677694 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:19.677849 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 13:09:19.678017 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":267,"threads_started":1}
W20260809 13:09:19.679030 13432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51962
W20260809 13:09:19.679669 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13227: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:19.679803 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13227
I20260809 13:09:19.711383 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12639
I20260809 13:09:19.712594 12752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:19.830112 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12639
2026-08-09T13:09:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4561 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[13596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[13596](info): set up 2 sockets
Aug 09 13:09:19 dist-test-slave-42z9 krb5kdc[13596](info): commencing operation
krb5kdc: starting...
W20260809 13:09:21.028371 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:21 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:21.226562 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:42429
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:21.545326 13611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:21.545603 13611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:21.545730 13611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:21.552701 13611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:21.552798 13611 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 13:09:21.552863 13611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:21.552942 13611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:21.553007 13611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:21.553098 13611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:21.562733 13611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:42429
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:21.563954 13611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:21.565275 13611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:21.576620 13623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:21.576697 13618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:21.576668 13620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:21.577658 13611 server_base.cc:1034] running on GCE node
I20260809 13:09:21.578385 13611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:21.579718 13611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:21.580942 13611 hybrid_clock.cc:648] HybridClock initialized: now 1786280961580885 us; error 27 us; skew 500 ppm
I20260809 13:09:21.581257 13611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:21 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280961, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:21.584190 13611 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:21.585686 13611 webserver.cc:460] Webserver started at http://127.10.181.190:44723/ using document root <none> and password file <none>
I20260809 13:09:21.586357 13611 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:21.586454 13611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:21.586752 13611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:21.589632 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "fd90a13aaeca418e872ce8799fb8455d"
format_stamp: "Formatted at 2026-08-09 13:09:21 on dist-test-slave-42z9"
server_key: "08d1fa5612f0890b27d929e3475fd0cb"
server_key_iv: "da9f0450689766fc1e3f49df2635f0b3"
server_key_version: "encryptionkey@0"
I20260809 13:09:21.590386 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "fd90a13aaeca418e872ce8799fb8455d"
format_stamp: "Formatted at 2026-08-09 13:09:21 on dist-test-slave-42z9"
server_key: "08d1fa5612f0890b27d929e3475fd0cb"
server_key_iv: "da9f0450689766fc1e3f49df2635f0b3"
server_key_version: "encryptionkey@0"
I20260809 13:09:21.595335 13611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 13:09:21.598698 13635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:21.599888 13611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:21.600047 13611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "fd90a13aaeca418e872ce8799fb8455d"
format_stamp: "Formatted at 2026-08-09 13:09:21 on dist-test-slave-42z9"
server_key: "08d1fa5612f0890b27d929e3475fd0cb"
server_key_iv: "da9f0450689766fc1e3f49df2635f0b3"
server_key_version: "encryptionkey@0"
I20260809 13:09:21.600250 13611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:21.612576 13611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:21.616202 13611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:21.616436 13611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:21.636382 13611 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:42429
I20260809 13:09:21.636395 13748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:42429 every 8 connection(s)
I20260809 13:09:21.637735 13611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:21.641474 13749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:21.641851 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13611
I20260809 13:09:21.642036 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:21.642658 10966 external_mini_cluster.cc:1428] Setting key 22fbd07c38daa3210df303c96d75fae1
I20260809 13:09:21.654776 13749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Bootstrap starting.
Aug 09 13:09:21 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:21.658354 13749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:21.659413 13749 log.cc:824] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:21.662130 13749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: No bootstrap required, opened a new log
I20260809 13:09:21.664727 13756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:21.644565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.10.181.190:42429)
0809 13:09:21.645338 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:21.645349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:21.645391 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:09:21.647330 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:21.647352 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:21.647733 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:21.648037 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:21.649802 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:21.651049 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:21.652766 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:21.653382 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:21.653549 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:21.657980 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:21.657988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:21.658001 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:21.658040 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:21.660893 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:21.661544 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:21.661550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:21.661557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:21.661737 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:21.662088 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:21.662092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:21.662094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:21.662653 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:21.662930 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:21.663371 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 13:09:21.663771 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":344,"threads_started":1}
I20260809 13:09:21.669785 13749 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } }
I20260809 13:09:21.670130 13749 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:21.670223 13749 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90a13aaeca418e872ce8799fb8455d, State: Initialized, Role: FOLLOWER
I20260809 13:09:21.670744 13749 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } }
I20260809 13:09:21.671012 13749 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:21.671178 13749 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:21.671370 13749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:21.673534 13749 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } }
I20260809 13:09:21.674008 13749 leader_election.cc:302] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd90a13aaeca418e872ce8799fb8455d; no voters:
I20260809 13:09:21.674639 13749 leader_election.cc:288] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:21.674724 13759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:21.675458 13759 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [term 1 LEADER]: Becoming Leader. State: Replica: fd90a13aaeca418e872ce8799fb8455d, State: Running, Role: LEADER
I20260809 13:09:21.676016 13759 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } }
I20260809 13:09:21.676791 13749 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:21.678668 13763 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd90a13aaeca418e872ce8799fb8455d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } } }
I20260809 13:09:21.678728 13761 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd90a13aaeca418e872ce8799fb8455d. Latest consensus state: current_term: 1 leader_uuid: "fd90a13aaeca418e872ce8799fb8455d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd90a13aaeca418e872ce8799fb8455d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 42429 } } }
I20260809 13:09:21.679029 13763 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:21.679032 13761 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:21.682301 13768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:21.687973 13768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:21.697664 13768 catalog_manager.cc:1347] Generated new cluster ID: d5eff808fe8142cbb3f4a0cf903fab5d
I20260809 13:09:21.697911 13768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:21.728785 13768 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:21.730190 13768 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:21.746487 13768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Generated new TSK 0
I20260809 13:09:21.747305 13768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:21.809859 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42429
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:22.139894 13792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:22.140156 13792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:22.140290 13792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:22.147264 13792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:22.147360 13792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:22.147480 13792 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 13:09:22.147543 13792 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:22.147639 13792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:22.156963 13792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:22.158145 13792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:22.159654 13792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:22.173244 13802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:22.173219 13804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:22.173513 13801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:22.174041 13792 server_base.cc:1034] running on GCE node
I20260809 13:09:22.174687 13792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:22.175562 13792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:22.176771 13792 hybrid_clock.cc:648] HybridClock initialized: now 1786280962176714 us; error 41 us; skew 500 ppm
I20260809 13:09:22.177094 13792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:22 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280962, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:22.180178 13792 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:22.181766 13792 webserver.cc:460] Webserver started at http://127.10.181.129:46369/ using document root <none> and password file <none>
I20260809 13:09:22.182524 13792 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:22.182631 13792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:22.182896 13792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:22.185827 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "b7af90a1faf045c4a746e140167f5871"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "085909d887c96f2f971c16668173bac8"
server_key_iv: "539cfbe0f090b187080a898b34e70a6d"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.186558 13792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "b7af90a1faf045c4a746e140167f5871"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "085909d887c96f2f971c16668173bac8"
server_key_iv: "539cfbe0f090b187080a898b34e70a6d"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.191479 13792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:22.194818 13816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:22.196090 13792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:22.196244 13792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "b7af90a1faf045c4a746e140167f5871"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "085909d887c96f2f971c16668173bac8"
server_key_iv: "539cfbe0f090b187080a898b34e70a6d"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.196414 13792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:22.222931 13792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:22.226675 13792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:22.226933 13792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:22.229585 13792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:22.229684 13792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:22.229801 13792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:22.229911 13792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:09:22 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280962, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:22.252269 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.230110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54821 (local address 127.10.181.190:42429)
0809 13:09:22.230814 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:22.230820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:22.231792 (+ 972us) server_negotiation.cc:407] Connection header received
0809 13:09:22.234544 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:22.234551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:22.234657 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:22.234779 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:22.237164 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.238093 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.240522 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.241187 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.241320 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.245536 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:22.245541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:22.245545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:22.245580 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:22.248552 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.249424 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.249429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.249433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.249631 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.250256 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.250261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.250264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.250571 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:22.250707 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:22.251608 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 13:09:22.251860 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":450,"threads_started":1}
I20260809 13:09:22.253309 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.230722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.129:54821)
0809 13:09:22.231600 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:22.231658 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:22.234233 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:22.235082 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:22.235097 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:22.235944 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:22.236934 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.236964 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.238302 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.238306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.240364 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.240387 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.241488 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.241497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.241780 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.242522 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:22.242549 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:22.245272 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:22.248696 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.248705 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.248716 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.249309 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.249764 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.249770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.249775 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.250117 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.250719 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:22.250727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:22.251182 (+ 455us) client_negotiation.cc:769] Sending connection context
0809 13:09:22.251496 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 13:09:22.252065 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":318,"threads_started":1}
I20260809 13:09:22.271220 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.259302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:41469 (local address 127.10.181.190:42429)
0809 13:09:22.259466 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:22.259470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:22.260119 (+ 649us) server_negotiation.cc:407] Connection header received
0809 13:09:22.260241 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:22.260247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:22.260329 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:22.260436 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:22.261273 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.262396 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.264275 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.264669 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.264774 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.266470 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:22.266475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:22.266478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:22.266508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:22.268676 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.269295 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.269300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.269303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.269426 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.269825 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.269830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.269834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.270117 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:22.270267 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:22.270658 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 13:09:22.270864 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:09:22.271296 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.259437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.129:41469)
0809 13:09:22.260018 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:22.260033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:22.260148 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:22.260442 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:22.260445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:22.260869 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:22.261157 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.261165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.262526 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.262535 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.264138 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.264151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.264848 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.264854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.264997 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.265593 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:22.265609 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:22.266299 (+ 690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:22.268797 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.268803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.268807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.269186 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.269565 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.269581 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.269585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.269715 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.270231 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:22.270236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:22.270411 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 13:09:22.270526 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 13:09:22.270846 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":292,"threads_started":1}
I20260809 13:09:22.281157 13792 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:38213
I20260809 13:09:22.281205 14008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:38213 every 8 connection(s)
I20260809 13:09:22.282368 13792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:22.284247 14009 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42429
I20260809 13:09:22.284466 14009 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:22.285040 14009 heartbeater.cc:511] Master 127.10.181.190:42429 requested a full tablet report, sending...
I20260809 13:09:22.286924 13670 ts_manager.cc:194] Registered new tserver with Master: b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213)
I20260809 13:09:22.288300 13670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:54821
I20260809 13:09:22.290800 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13792
I20260809 13:09:22.290964 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:22.291530 10966 external_mini_cluster.cc:1428] Setting key 227323f2ade34505bd363c4cab5990e2
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:22.334137 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42429
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:22.656633 14014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:22.656905 14014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:22.657048 14014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:22.664278 14014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:22.664399 14014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:22.664527 14014 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 13:09:22.664590 14014 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:22.664698 14014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:22.674362 14014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:22.675601 14014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:22.676911 14014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:22.689366 14031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:22.689420 14027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:22.689666 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:22.690270 14014 server_base.cc:1034] running on GCE node
I20260809 13:09:22.690831 14014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:22.691651 14014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:22.692845 14014 hybrid_clock.cc:648] HybridClock initialized: now 1786280962692798 us; error 39 us; skew 500 ppm
I20260809 13:09:22.693137 14014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:22 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280962, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:22.698256 14014 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:22.699632 14014 webserver.cc:460] Webserver started at http://127.10.181.130:34503/ using document root <none> and password file <none>
I20260809 13:09:22.700255 14014 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:22.700371 14014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:22.700687 14014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:22.703537 14014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "344c262ec438447b88a371ea37661be9"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "4eb9716626c051d98f6785a1ede1da9d"
server_key_iv: "dc19cf75e4df8a4f98268cb480191ffe"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.704344 14014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "344c262ec438447b88a371ea37661be9"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "4eb9716626c051d98f6785a1ede1da9d"
server_key_iv: "dc19cf75e4df8a4f98268cb480191ffe"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.709136 14014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 13:09:22.712356 14044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:22.713652 14014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:09:22.713802 14014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "344c262ec438447b88a371ea37661be9"
format_stamp: "Formatted at 2026-08-09 13:09:22 on dist-test-slave-42z9"
server_key: "4eb9716626c051d98f6785a1ede1da9d"
server_key_iv: "dc19cf75e4df8a4f98268cb480191ffe"
server_key_version: "encryptionkey@0"
I20260809 13:09:22.713981 14014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:22.725922 14014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:22.729630 14014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:22.729889 14014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:22.732364 14014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:22.732463 14014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:22.732580 14014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:22.732661 14014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:09:22 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280962, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:22.754354 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.732990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:37919 (local address 127.10.181.190:42429)
0809 13:09:22.733216 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:22.733223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:22.734452 (+ 1229us) server_negotiation.cc:407] Connection header received
0809 13:09:22.736441 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:22.736447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:22.736540 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:22.736659 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:22.738839 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.739803 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.742604 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.743146 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.743266 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.747767 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:22.747774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:22.747780 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:22.747825 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:22.750892 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.751845 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.751851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.751855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.752028 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.752501 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.752506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.752509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.752881 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:22.753049 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:22.753774 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 13:09:22.754019 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:09:22.755625 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.733410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.130:37919)
0809 13:09:22.734268 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:22.734314 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:22.736143 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:22.736960 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:22.736969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:22.737784 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:22.738604 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.738633 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.739993 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.740003 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.742440 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.742457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.743453 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.743461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.743768 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.744620 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:22.744646 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:22.747528 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:22.751030 (+ 3502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.751040 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.751066 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.751691 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.752161 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.752165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.752167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.752369 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.752990 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:22.752995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:22.753310 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 13:09:22.753637 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 13:09:22.754162 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":344,"threads_started":1}
I20260809 13:09:22.780638 14014 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43025
I20260809 13:09:22.780692 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43025 every 8 connection(s)
I20260809 13:09:22.781463 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.767530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:58541 (local address 127.10.181.190:42429)
0809 13:09:22.767713 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:22.767719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:22.768369 (+ 650us) server_negotiation.cc:407] Connection header received
0809 13:09:22.768564 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:22.768568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:22.768632 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:22.768711 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:22.769881 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.770622 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.772579 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.773032 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.773160 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.775496 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:22.775502 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:22.775506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:22.775545 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:22.778369 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.779100 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.779106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.779110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.779246 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:22.779689 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:22.779694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:22.779696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:22.780089 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:22.780240 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:22.780935 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 13:09:22.781158 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:09:22.781495 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:22.767542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.130:58541)
0809 13:09:22.768252 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:22.768273 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:22.768417 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:22.768753 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:22.768758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:22.769333 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:22.769732 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.769746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.770743 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.770749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.772423 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:22.772444 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:22.773383 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:22.773387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:22.773586 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:22.774262 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:22.774279 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:22.775320 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:22.778547 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.778553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.778560 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.778990 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.779350 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:22.779355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:22.779357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:22.779531 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:22.780308 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:22.780313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:22.780518 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 13:09:22.780655 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 13:09:22.780979 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":365,"threads_started":1}
I20260809 13:09:22.782107 14014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:22.784457 14232 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42429
I20260809 13:09:22.784734 14232 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:22.785434 14232 heartbeater.cc:511] Master 127.10.181.190:42429 requested a full tablet report, sending...
I20260809 13:09:22.786742 13670 ts_manager.cc:194] Registered new tserver with Master: 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:22.787513 13670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:37919
I20260809 13:09:22.791544 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14014
I20260809 13:09:22.791685 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:22.792116 10966 external_mini_cluster.cc:1428] Setting key 64935b4c0cea7bf3a54daf8bc7cbf0b7
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:22.835665 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:42429
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:23.163797 14240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:23.164049 14240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:23.164155 14240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:23.171228 14240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:23.171326 14240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:23.171422 14240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 13:09:23.171473 14240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:09:23.171538 14240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:23.180809 14240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:23.182031 14240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:23.183245 14240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:23.195472 14248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:23.195499 14249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:23.195428 14252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:23.196712 14240 server_base.cc:1034] running on GCE node
I20260809 13:09:23.197252 14240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:23.198124 14240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:23.199306 14240 hybrid_clock.cc:648] HybridClock initialized: now 1786280963199251 us; error 48 us; skew 500 ppm
I20260809 13:09:23.199635 14240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:23 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280963, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:23.202610 14240 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:23.203984 14240 webserver.cc:460] Webserver started at http://127.10.181.131:34027/ using document root <none> and password file <none>
I20260809 13:09:23.204649 14240 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:23.204768 14240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:23.205068 14240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:23.207805 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
format_stamp: "Formatted at 2026-08-09 13:09:23 on dist-test-slave-42z9"
server_key: "b04c2409cbfc00fba6126ada9887d9fd"
server_key_iv: "7d244d642ab016cfab21675a963ee2d8"
server_key_version: "encryptionkey@0"
I20260809 13:09:23.208513 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
format_stamp: "Formatted at 2026-08-09 13:09:23 on dist-test-slave-42z9"
server_key: "b04c2409cbfc00fba6126ada9887d9fd"
server_key_iv: "7d244d642ab016cfab21675a963ee2d8"
server_key_version: "encryptionkey@0"
I20260809 13:09:23.213332 14240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 13:09:23.216573 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:23.217837 14240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:23.218001 14240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
format_stamp: "Formatted at 2026-08-09 13:09:23 on dist-test-slave-42z9"
server_key: "b04c2409cbfc00fba6126ada9887d9fd"
server_key_iv: "7d244d642ab016cfab21675a963ee2d8"
server_key_version: "encryptionkey@0"
I20260809 13:09:23.218190 14240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:23.254748 14240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:23.258860 14240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:23.259132 14240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:23.261679 14240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:23.261780 14240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:23.261896 14240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:23.261973 14240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:09:23 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280963, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:23.283780 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.262355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:34121 (local address 127.10.181.190:42429)
0809 13:09:23.262555 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.262561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.263908 (+ 1347us) server_negotiation.cc:407] Connection header received
0809 13:09:23.266085 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.266092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.266194 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.266284 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:23.268444 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.269623 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.271882 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.272413 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.272539 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.277104 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:23.277109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:23.277112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:23.277149 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:23.280385 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.281176 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.281182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.281185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.281322 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.281864 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.281869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.281872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.282177 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:23.282329 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.283160 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.283388 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:09:23.284688 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.262885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.131:34121)
0809 13:09:23.263725 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.263774 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.265801 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.266554 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.266569 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.267360 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:23.268174 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.268198 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.269808 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.269814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.271742 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.271755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.272850 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.272856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.273116 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.273813 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:23.273849 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:23.276816 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:23.280564 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.280573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.280592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.281053 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.281431 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.281463 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.281467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.281718 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.282318 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:23.282326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:23.282673 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.282929 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.283416 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":357,"threads_started":1}
I20260809 13:09:23.290890 14009 heartbeater.cc:503] Master 127.10.181.190:42429 was elected leader, sending a full tablet report...
I20260809 13:09:23.309813 14240 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:36037
I20260809 13:09:23.309861 14449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:36037 every 8 connection(s)
I20260809 13:09:23.310438 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.295786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:51521 (local address 127.10.181.190:42429)
0809 13:09:23.295964 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.295969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.296597 (+ 628us) server_negotiation.cc:407] Connection header received
0809 13:09:23.296762 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.296765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.296823 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.296898 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:23.298316 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.299389 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.301638 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.302239 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.302337 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.304667 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:23.304674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:23.304680 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:23.304721 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:23.307858 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.308497 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.308503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.308506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.308639 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.308995 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.309000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.309003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.309276 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:23.309424 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.309874 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.310085 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:09:23.310572 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.295865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.10.181.131:51521)
0809 13:09:23.296491 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.296511 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.296658 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.296938 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.296943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.297538 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:23.298120 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.298137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.299623 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.299629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.301416 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.301455 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.302640 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.302647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.302854 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.303599 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:23.303622 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:23.304445 (+ 823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:23.307980 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.307986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.307990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.308376 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.308753 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.308758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.308763 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.308887 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.309405 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:23.309410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:23.309618 (+ 208us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.309755 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.310140 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":319,"threads_started":1}
I20260809 13:09:23.311306 14240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:23.313493 14450 heartbeater.cc:347] Connected to a master server at 127.10.181.190:42429
I20260809 13:09:23.313750 14450 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:23.314560 14450 heartbeater.cc:511] Master 127.10.181.190:42429 requested a full tablet report, sending...
I20260809 13:09:23.315770 13670 ts_manager.cc:194] Registered new tserver with Master: 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037)
I20260809 13:09:23.316515 13670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:34121
I20260809 13:09:23.318410 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14240
I20260809 13:09:23.318564 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:23.319003 10966 external_mini_cluster.cc:1428] Setting key 9a660e23e1d62ad18c3840f0b2adf3d7
I20260809 13:09:23.321697 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 13:09:23 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 13:09:23 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:23.370872 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.358197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.10.181.190:42429)
0809 13:09:23.358417 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.358423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.358593 (+ 170us) server_negotiation.cc:407] Connection header received
0809 13:09:23.358795 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.358800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.358865 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.358950 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:23.360300 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.361036 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.362318 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.362730 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.362843 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.366371 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:23.366376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:23.366379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:23.366413 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:23.368589 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.369180 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.369185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.369187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.369355 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.369745 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.369749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.369751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.370018 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:23.370212 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.370439 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.370599 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:09:23.374694 13670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:23.381563 13670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:23.406422 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.393010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:36037 (local address 127.0.0.1:58942)
0809 13:09:23.394516 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.394535 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.394649 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.398931 (+ 4282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.398938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.398969 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.399606 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.399623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401541 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.401547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.404031 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.404050 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.405390 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.405397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.405626 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.405678 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.405816 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.405987 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1371,"mutex_wait_us":64,"thread_start_us":1248,"threads_started":1}
I20260809 13:09:23.406509 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.392976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.0.0.1:33466)
0809 13:09:23.393736 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.393789 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.393952 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.399099 (+ 5147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.399104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.399126 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.399788 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.399806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401766 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.401772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.404105 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.404116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.405763 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.405769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.405915 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.405945 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.406032 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.406159 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":31,"spinlock_wait_cycles":17664,"thread_start_us":340,"threads_started":1}
I20260809 13:09:23.406930 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.394153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43025 (local address 127.0.0.1:60968)
0809 13:09:23.397895 (+ 3742us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.397913 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.398113 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.399002 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.399007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.399038 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.399826 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.399841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401803 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.401817 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.403912 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.403929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.406047 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.406054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.406238 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.406279 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.406369 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.406504 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3625,"mutex_wait_us":60,"spinlock_wait_cycles":1792,"thread_start_us":3465,"threads_started":1}
I20260809 13:09:23.407083 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.393041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.10.181.129:38213)
0809 13:09:23.398538 (+ 5497us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.398547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.398572 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:09:23.398692 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.398699 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.398894 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.399258 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.399944 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401625 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.404227 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.405580 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.405734 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.405876 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.406629 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.406791 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5345,"thread_start_us":5990,"threads_started":1}
I20260809 13:09:23.407517 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.394714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.10.181.130:43025)
0809 13:09:23.395335 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.395345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.398012 (+ 2667us) server_negotiation.cc:407] Connection header received
0809 13:09:23.398336 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.398344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.398579 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.398755 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.400059 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401654 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.404063 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.405523 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.405710 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.405876 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.407138 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.407311 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":996,"threads_started":1}
I20260809 13:09:23.408823 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.393736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.10.181.131:36037)
0809 13:09:23.398296 (+ 4560us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.398305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.398337 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:09:23.398434 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.398442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.398686 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.399852 (+ 1166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.400182 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.401390 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.404231 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.405222 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.405394 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.405592 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.407598 (+ 2006us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.408483 (+ 885us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4389,"thread_start_us":4221,"threads_started":1}
I20260809 13:09:23.410583 14127 tablet_service.cc:1467] Processing CreateTablet for tablet 7104b7caf7714677bfb8f1c8662388fd (DEFAULT_TABLE table=test-table [id=722f81a629bf45279a1f389077d06282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.410573 13897 tablet_service.cc:1467] Processing CreateTablet for tablet 7104b7caf7714677bfb8f1c8662388fd (DEFAULT_TABLE table=test-table [id=722f81a629bf45279a1f389077d06282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.411943 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7104b7caf7714677bfb8f1c8662388fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.412021 13897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7104b7caf7714677bfb8f1c8662388fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.412148 14344 tablet_service.cc:1467] Processing CreateTablet for tablet 7104b7caf7714677bfb8f1c8662388fd (DEFAULT_TABLE table=test-table [id=722f81a629bf45279a1f389077d06282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.413468 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7104b7caf7714677bfb8f1c8662388fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.422840 14482 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Bootstrap starting.
I20260809 13:09:23.426028 14483 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Bootstrap starting.
I20260809 13:09:23.426210 14482 tablet_bootstrap.cc:654] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.427520 14482 log.cc:824] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:23.429361 14483 tablet_bootstrap.cc:654] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.430614 14483 log.cc:824] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:23.430711 14482 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: No bootstrap required, opened a new log
I20260809 13:09:23.430958 14482 ts_tablet_manager.cc:1397] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 13:09:23.431428 14485 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Bootstrap starting.
I20260809 13:09:23.434526 14485 tablet_bootstrap.cc:654] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.435468 14485 log.cc:824] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:23.438267 14483 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: No bootstrap required, opened a new log
I20260809 13:09:23.438447 14485 tablet_bootstrap.cc:492] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: No bootstrap required, opened a new log
I20260809 13:09:23.438515 14483 ts_tablet_manager.cc:1397] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260809 13:09:23.438700 14485 ts_tablet_manager.cc:1397] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 13:09:23.441594 14482 raft_consensus.cc:348] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.442019 14482 raft_consensus.cc:374] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.442126 14482 raft_consensus.cc:729] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344c262ec438447b88a371ea37661be9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.442767 14482 consensus_queue.cc:260] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.443861 14232 heartbeater.cc:503] Master 127.10.181.190:42429 was elected leader, sending a full tablet report...
I20260809 13:09:23.444484 14482 ts_tablet_manager.cc:1428] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 13:09:23.446846 14483 raft_consensus.cc:348] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.447177 14483 raft_consensus.cc:374] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.447278 14483 raft_consensus.cc:729] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.447813 14483 consensus_queue.cc:260] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.449607 14483 ts_tablet_manager.cc:1428] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 13:09:23.449761 14485 raft_consensus.cc:348] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.450335 14485 raft_consensus.cc:374] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.450433 14485 raft_consensus.cc:729] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.450964 14485 consensus_queue.cc:260] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.451956 14450 heartbeater.cc:503] Master 127.10.181.190:42429 was elected leader, sending a full tablet report...
I20260809 13:09:23.452607 14485 ts_tablet_manager.cc:1428] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 13:09:23.527323 14492 raft_consensus.cc:482] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:23.527626 14492 raft_consensus.cc:504] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.529359 14492 leader_election.cc:288] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
W20260809 13:09:23.534145 14236 tablet.cc:2369] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:09:23.534552 14010 tablet.cc:2369] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:09:23.536502 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.529947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:40115 (local address 127.10.181.130:43025)
0809 13:09:23.530182 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.530189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.530815 (+ 626us) server_negotiation.cc:407] Connection header received
0809 13:09:23.530975 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.530980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.531056 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.531136 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.531764 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.532511 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.534691 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.535642 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.535796 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.535870 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.536137 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.536261 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:09:23.536652 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.529716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43025 (local address 127.10.181.129:40115)
0809 13:09:23.530705 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.530720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.530840 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.531169 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.531174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.531207 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.531611 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.531624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.532653 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.532658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.534487 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.534508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.535842 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.535848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.535985 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.536009 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.536092 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.536234 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":53,"thread_start_us":671,"threads_started":1}
I20260809 13:09:23.536809 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.529818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:43045 (local address 127.10.181.131:36037)
0809 13:09:23.530022 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.530028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.530939 (+ 911us) server_negotiation.cc:407] Connection header received
0809 13:09:23.531096 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.531099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.531174 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.531269 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.531868 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.532546 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.535214 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.535959 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.536112 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.536187 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.536453 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.536587 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:09:23.537096 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.529765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:36037 (local address 127.10.181.129:43045)
0809 13:09:23.530819 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.530837 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.530974 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.531302 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.531308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.531338 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.531709 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.531720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.532684 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.532688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.535061 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.535081 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.536176 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.536182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.536306 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.536330 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.536412 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.536530 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":52,"thread_start_us":560,"threads_started":1}
I20260809 13:09:23.537773 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7104b7caf7714677bfb8f1c8662388fd" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9" is_pre_election: true
I20260809 13:09:23.538051 14377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7104b7caf7714677bfb8f1c8662388fd" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" is_pre_election: true
I20260809 13:09:23.538354 14159 raft_consensus.cc:2454] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 0.
I20260809 13:09:23.538590 14377 raft_consensus.cc:2454] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 0.
I20260809 13:09:23.539032 13820 leader_election.cc:302] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.539433 14492 raft_consensus.cc:2781] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:23.539561 14492 raft_consensus.cc:482] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:23.539659 14492 raft_consensus.cc:3037] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.541909 14492 raft_consensus.cc:504] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.542557 14492 leader_election.cc:288] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 election: Requested vote from peers 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:23.542928 14377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7104b7caf7714677bfb8f1c8662388fd" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
I20260809 13:09:23.543047 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7104b7caf7714677bfb8f1c8662388fd" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9"
I20260809 13:09:23.543140 14377 raft_consensus.cc:3037] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.543251 14159 raft_consensus.cc:3037] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.546329 14159 raft_consensus.cc:2454] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 1.
I20260809 13:09:23.546329 14377 raft_consensus.cc:2454] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 1.
I20260809 13:09:23.546717 13821 leader_election.cc:302] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f80fb4ab8cf4eb384e8e56a749975a9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.547010 14492 raft_consensus.cc:2781] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:23.547428 14492 raft_consensus.cc:686] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 1 LEADER]: Becoming Leader. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Running, Role: LEADER
I20260809 13:09:23.547865 14492 consensus_queue.cc:237] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } }
I20260809 13:09:23.552440 13670 catalog_manager.cc:5495] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7af90a1faf045c4a746e140167f5871 (127.10.181.129). New cstate: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } health_report { overall_health: UNKNOWN } } }
W20260809 13:09:23.562915 14452 tablet.cc:2369] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 13:09:23 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280961, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:23.620250 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.607699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:42429 (local address 127.0.0.1:43246)
0809 13:09:23.608236 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.608251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.608385 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.608679 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.608683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.609379 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:23.609747 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.609758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.610764 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.610768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.611724 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.611737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.612490 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.612495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.612653 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.613214 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:23.613236 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:23.615336 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:23.618018 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.618029 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.618036 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.618438 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.618765 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:23.618769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:23.618771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:23.618912 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:23.619399 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:23.619405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:23.619565 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.619708 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.619907 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":274,"threads_started":1}
I20260809 13:09:23.620288 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.607633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.10.181.190:42429)
0809 13:09:23.607815 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.607824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.608337 (+ 513us) server_negotiation.cc:407] Connection header received
0809 13:09:23.608493 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.608496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.608552 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.608631 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:23.609895 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.610636 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.611865 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.612294 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.612406 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.615509 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:23.615514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:23.615517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:23.615545 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:23.617906 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.618543 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.618547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.618549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.618671 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:23.618999 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:23.619002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:23.619004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:23.619282 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:23.619404 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.619803 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.620021 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":55}
I20260809 13:09:23.625308 13670 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:43246:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 13:09:23.626379 13670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:23.632941 14127 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0f8f5e92f64996a6ae0891395b72d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25ced6bc770a4d2b88805468130e2e5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:23.632864 13897 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0f8f5e92f64996a6ae0891395b72d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25ced6bc770a4d2b88805468130e2e5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:23.633270 14344 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0f8f5e92f64996a6ae0891395b72d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25ced6bc770a4d2b88805468130e2e5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:09:23.633852 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0f8f5e92f64996a6ae0891395b72d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.633996 13897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0f8f5e92f64996a6ae0891395b72d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.634096 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0f8f5e92f64996a6ae0891395b72d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.640713 14482 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: Bootstrap starting.
I20260809 13:09:23.641062 14483 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: Bootstrap starting.
I20260809 13:09:23.641867 14485 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: Bootstrap starting.
I20260809 13:09:23.642506 14482 tablet_bootstrap.cc:654] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.642678 14483 tablet_bootstrap.cc:654] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.643436 14485 tablet_bootstrap.cc:654] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.654474 14482 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: No bootstrap required, opened a new log
I20260809 13:09:23.654703 14482 ts_tablet_manager.cc:1397] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260809 13:09:23.654812 14483 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: No bootstrap required, opened a new log
I20260809 13:09:23.654863 14485 tablet_bootstrap.cc:492] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: No bootstrap required, opened a new log
I20260809 13:09:23.655042 14483 ts_tablet_manager.cc:1397] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260809 13:09:23.655187 14485 ts_tablet_manager.cc:1397] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260809 13:09:23.655826 14482 raft_consensus.cc:348] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.655825 14483 raft_consensus.cc:348] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.656150 14483 raft_consensus.cc:374] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.656160 14482 raft_consensus.cc:374] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.656253 14483 raft_consensus.cc:729] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.655983 14485 raft_consensus.cc:348] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.656350 14482 raft_consensus.cc:729] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344c262ec438447b88a371ea37661be9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.656477 14485 raft_consensus.cc:374] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.656585 14485 raft_consensus.cc:729] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.656569 14483 consensus_queue.cc:260] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.656949 14485 consensus_queue.cc:260] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.657049 14492 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.657392 14492 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.657239 14483 ts_tablet_manager.cc:1428] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 13:09:23.657634 14485 ts_tablet_manager.cc:1428] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:09:23.657596 14494 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.657984 14494 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.656761 14482 consensus_queue.cc:260] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.659106 14491 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.659559 14491 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.661041 14482 ts_tablet_manager.cc:1428] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 13:09:23.704926 14492 raft_consensus.cc:482] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:23.705159 14492 raft_consensus.cc:504] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.705960 14492 leader_election.cc:288] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025), 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037)
I20260809 13:09:23.706414 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0f8f5e92f64996a6ae0891395b72d6" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9" is_pre_election: true
I20260809 13:09:23.706511 14377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0f8f5e92f64996a6ae0891395b72d6" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" is_pre_election: true
I20260809 13:09:23.706677 14159 raft_consensus.cc:2454] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 0.
I20260809 13:09:23.706769 14377 raft_consensus.cc:2454] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 0.
I20260809 13:09:23.707090 13820 leader_election.cc:302] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.707391 14492 raft_consensus.cc:2781] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:23.707486 14492 raft_consensus.cc:482] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:23.707584 14492 raft_consensus.cc:3037] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.709718 14492 raft_consensus.cc:504] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.710304 14492 leader_election.cc:288] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 election: Requested vote from peers 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025), 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037)
I20260809 13:09:23.710695 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0f8f5e92f64996a6ae0891395b72d6" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9"
I20260809 13:09:23.710767 14377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0f8f5e92f64996a6ae0891395b72d6" candidate_uuid: "b7af90a1faf045c4a746e140167f5871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
I20260809 13:09:23.710891 14159 raft_consensus.cc:3037] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.710969 14377 raft_consensus.cc:3037] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.712858 14159 raft_consensus.cc:2454] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 1.
I20260809 13:09:23.712934 14377 raft_consensus.cc:2454] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7af90a1faf045c4a746e140167f5871 in term 1.
I20260809 13:09:23.713233 13820 leader_election.cc:302] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.713485 14492 raft_consensus.cc:2781] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:23.713675 14492 raft_consensus.cc:686] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [term 1 LEADER]: Becoming Leader. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Running, Role: LEADER
I20260809 13:09:23.713907 14492 consensus_queue.cc:237] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.715539 14497 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: TxnStatusTablet state changed. Reason: New leader b7af90a1faf045c4a746e140167f5871. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.715860 14497 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:23.716508 14514 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 13:09:23.716836 14514 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 13:09:23.717630 13667 catalog_manager.cc:5495] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7af90a1faf045c4a746e140167f5871 (127.10.181.129). New cstate: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } health_report { overall_health: UNKNOWN } } }
I20260809 13:09:23.749640 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.743614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.10.181.129:38213)
0809 13:09:23.743843 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.743848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.743944 (+ 96us) server_negotiation.cc:407] Connection header received
0809 13:09:23.744219 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.744223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.744304 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.744422 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:23.744822 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.745611 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.747494 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.747930 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.748023 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.748544 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:23.749050 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:23.749161 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.749299 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.749370 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:09:23.749773 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.743586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.0.0.1:33470)
0809 13:09:23.743857 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.743878 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.744031 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.744465 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.744470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.744484 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:23.744687 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.744698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.745732 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.745739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.747328 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.747344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.748090 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.748097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.748295 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.748368 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:23.749157 (+ 789us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:23.749180 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.749275 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.749431 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 13:09:23.757474 14377 raft_consensus.cc:1261] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Refusing update from remote peer b7af90a1faf045c4a746e140167f5871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:23.757521 14159 raft_consensus.cc:1261] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Refusing update from remote peer b7af90a1faf045c4a746e140167f5871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:23.758497 14497 consensus_queue.cc:1035] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:23.758841 14492 consensus_queue.cc:1035] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:23.766013 14494 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: TxnStatusTablet state changed. Reason: New leader b7af90a1faf045c4a746e140167f5871. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.766331 14494 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.770510 14518 mvcc.cc:205] Tried to move back new op lower bound from 7316606827540467712 to 7316606827374477312. Current Snapshot: MvccSnapshot[applied={T|T < 7316606827540467712}]
I20260809 13:09:23.775399 14497 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.776300 14497 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:23.777204 14491 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: TxnStatusTablet state changed. Reason: New leader b7af90a1faf045c4a746e140167f5871. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.777529 14491 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.782693 14494 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.782927 14494 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 4f80fb4ab8cf4eb384e8e56a749975a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.789800 14497 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.790098 14497 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P b7af90a1faf045c4a746e140167f5871: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:09:23.795207 14491 tablet_replica.cc:439] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7af90a1faf045c4a746e140167f5871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } } }
I20260809 13:09:23.795465 14491 tablet_replica.cc:442] T 9e0f8f5e92f64996a6ae0891395b72d6 P 344c262ec438447b88a371ea37661be9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:09:23.802342 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.797097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.10.181.129:38213)
0809 13:09:23.797268 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.797275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.797516 (+ 241us) server_negotiation.cc:407] Connection header received
0809 13:09:23.797726 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.797729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.797793 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.797865 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:23.798251 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.799047 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.800563 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.801031 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.801149 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.801595 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:23.801729 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:23.801805 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.801952 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.802030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:09:23.820363 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.813030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:56949 (local address 127.10.181.129:38213)
0809 13:09:23.813413 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.813420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.814596 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 13:09:23.814869 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.814875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.814962 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.815066 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:23.815605 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.816779 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.818392 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.819087 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.819217 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.819657 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:23.819784 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:23.819877 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.819984 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.820068 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260809 13:09:23.820386 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.813071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.10.181.129:56949)
0809 13:09:23.814511 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.814528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.814702 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.815087 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.815090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.815101 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:23.815446 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.815455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.816901 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.816906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.818251 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.818268 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.819261 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.819265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.819394 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.819450 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:23.819880 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:23.819889 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.819948 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.820053 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":374,"threads_started":1}
I20260809 13:09:23.830476 14159 raft_consensus.cc:1261] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Refusing update from remote peer b7af90a1faf045c4a746e140167f5871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:23.830514 14377 raft_consensus.cc:1261] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Refusing update from remote peer b7af90a1faf045c4a746e140167f5871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:23.830998 14497 consensus_queue.cc:1035] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:23.831302 14529 consensus_queue.cc:1035] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:23.890529 13669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:23.890826 13669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:23.894693 13669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Sending DeleteTablet for 3 replicas of tablet 7104b7caf7714677bfb8f1c8662388fd
I20260809 13:09:23.895740 14344 tablet_service.cc:1514] Processing DeleteTablet for tablet 7104b7caf7714677bfb8f1c8662388fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:58942
I20260809 13:09:23.895781 14127 tablet_service.cc:1514] Processing DeleteTablet for tablet 7104b7caf7714677bfb8f1c8662388fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260809 13:09:23.897024 13897 tablet_service.cc:1514] Processing DeleteTablet for tablet 7104b7caf7714677bfb8f1c8662388fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33466
I20260809 13:09:23.897146 13669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:23.897881 13669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:23.900605 14543 tablet_replica.cc:330] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: stopping tablet replica
I20260809 13:09:23.901058 14543 raft_consensus.cc:2229] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:23.903225 14543 raft_consensus.cc:2258] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:23.905475 14543 ts_tablet_manager.cc:1905] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:23.905686 13896 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (DEFAULT_TABLE table=test-table [id=74af7cc2edf844b1845d7266c18f760c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.906061 14126 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (DEFAULT_TABLE table=test-table [id=74af7cc2edf844b1845d7266c18f760c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.906260 13896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.906611 14126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.906543 14344 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (DEFAULT_TABLE table=test-table [id=74af7cc2edf844b1845d7266c18f760c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:23.906934 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:23.913156 14482 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Bootstrap starting.
I20260809 13:09:23.913430 14483 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Bootstrap starting.
I20260809 13:09:23.914873 14482 tablet_bootstrap.cc:654] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.915150 14483 tablet_bootstrap.cc:654] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.916280 14485 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Bootstrap starting.
I20260809 13:09:23.918066 14485 tablet_bootstrap.cc:654] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:23.928727 14483 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: No bootstrap required, opened a new log
I20260809 13:09:23.928922 14483 ts_tablet_manager.cc:1397] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260809 13:09:23.930745 14485 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: No bootstrap required, opened a new log
I20260809 13:09:23.930831 14483 raft_consensus.cc:348] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.931000 14485 ts_tablet_manager.cc:1397] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.006s
I20260809 13:09:23.931125 14483 raft_consensus.cc:374] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.931228 14483 raft_consensus.cc:729] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.931322 14482 tablet_bootstrap.cc:492] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: No bootstrap required, opened a new log
I20260809 13:09:23.931485 14482 ts_tablet_manager.cc:1397] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.004s
I20260809 13:09:23.931519 14483 consensus_queue.cc:260] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.932181 14483 ts_tablet_manager.cc:1428] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 13:09:23.932133 14485 raft_consensus.cc:348] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.932454 14485 raft_consensus.cc:374] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.932578 14485 raft_consensus.cc:729] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.932910 14485 consensus_queue.cc:260] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.932986 14482 raft_consensus.cc:348] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.933233 14482 raft_consensus.cc:374] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:23.933334 14482 raft_consensus.cc:729] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344c262ec438447b88a371ea37661be9, State: Initialized, Role: FOLLOWER
I20260809 13:09:23.933663 14482 consensus_queue.cc:260] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.934612 14485 ts_tablet_manager.cc:1428] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 13:09:23.935738 14546 tablet_replica.cc:330] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: stopping tablet replica
I20260809 13:09:23.936193 14546 raft_consensus.cc:2229] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:23.936863 14546 raft_consensus.cc:2258] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:23.937390 14491 raft_consensus.cc:482] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:23.937664 14491 raft_consensus.cc:504] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.938426 14544 tablet_replica.cc:330] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: stopping tablet replica
I20260809 13:09:23.938815 14544 raft_consensus.cc:2229] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:23.939175 14544 raft_consensus.cc:2258] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:23.940905 14491 leader_election.cc:288] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037)
I20260809 13:09:23.941406 14546 ts_tablet_manager.cc:1905] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:23.946748 14482 ts_tablet_manager.cc:1428] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Time spent starting tablet: real 0.015s user 0.000s sys 0.002s
I20260809 13:09:23.948958 14546 ts_tablet_manager.cc:1918] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:23.949126 14546 log.cc:1196] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/7104b7caf7714677bfb8f1c8662388fd
I20260809 13:09:23.950778 14544 ts_tablet_manager.cc:1905] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:23.951682 14543 ts_tablet_manager.cc:1918] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:23.951877 14543 log.cc:1196] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/7104b7caf7714677bfb8f1c8662388fd
I20260809 13:09:23.952440 14543 ts_tablet_manager.cc:1939] T 7104b7caf7714677bfb8f1c8662388fd P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting consensus metadata
I20260809 13:09:23.958106 13640 catalog_manager.cc:4841] TS 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037): tablet 7104b7caf7714677bfb8f1c8662388fd (table test-table [id=722f81a629bf45279a1f389077d06282]) successfully deleted
I20260809 13:09:23.958841 14546 ts_tablet_manager.cc:1939] T 7104b7caf7714677bfb8f1c8662388fd P b7af90a1faf045c4a746e140167f5871: Deleting consensus metadata
I20260809 13:09:23.960474 13638 catalog_manager.cc:4841] TS b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213): tablet 7104b7caf7714677bfb8f1c8662388fd (table test-table [id=722f81a629bf45279a1f389077d06282]) successfully deleted
I20260809 13:09:23.960816 14544 ts_tablet_manager.cc:1918] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:23.961009 14544 log.cc:1196] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/7104b7caf7714677bfb8f1c8662388fd
I20260809 13:09:23.961530 14544 ts_tablet_manager.cc:1939] T 7104b7caf7714677bfb8f1c8662388fd P 344c262ec438447b88a371ea37661be9: Deleting consensus metadata
I20260809 13:09:23.963195 13638 catalog_manager.cc:4841] TS 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025): tablet 7104b7caf7714677bfb8f1c8662388fd (table test-table [id=722f81a629bf45279a1f389077d06282]) successfully deleted
I20260809 13:09:23.965725 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.941392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:40241 (local address 127.10.181.129:38213)
0809 13:09:23.941591 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.941598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.955639 (+ 14041us) server_negotiation.cc:407] Connection header received
0809 13:09:23.955946 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.955953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.956031 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.956094 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.958084 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.959192 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.963104 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.964398 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.964535 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.964627 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.965253 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.965401 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 13:09:23.965806 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.941059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.10.181.130:40241)
0809 13:09:23.955514 (+ 14455us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.955538 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.955764 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.956382 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.956388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.956418 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.956928 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.956948 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.959753 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.959758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.962886 (+ 3128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.962904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.964839 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.964849 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.965043 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.965078 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.965176 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.965305 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14315,"thread_start_us":20682,"threads_started":1}
I20260809 13:09:23.966723 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0b2854dc914cd5a4bc5a1e0db2b886" candidate_uuid: "344c262ec438447b88a371ea37661be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871" is_pre_election: true
I20260809 13:09:23.967151 13938 raft_consensus.cc:2454] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344c262ec438447b88a371ea37661be9 in term 0.
I20260809 13:09:23.967767 14047 leader_election.cc:302] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.968173 14491 raft_consensus.cc:2781] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:23.968300 14491 raft_consensus.cc:482] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:23.968411 14491 raft_consensus.cc:3037] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.969156 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.941708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:50237 (local address 127.10.181.131:36037)
0809 13:09:23.941870 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:23.941877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:23.962124 (+ 20247us) server_negotiation.cc:407] Connection header received
0809 13:09:23.962248 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:23.962253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:23.962324 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:23.962403 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:23.963210 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.964271 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.966796 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.967991 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.968136 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.968206 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:23.968748 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 13:09:23.968881 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:23.969247 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:23.941362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:36037 (local address 127.10.181.130:50237)
0809 13:09:23.962009 (+ 20647us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:23.962028 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:23.962140 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:23.962564 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:23.962569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:23.962592 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:23.963026 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.963037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.964492 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.964497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.966608 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:23.966623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:23.968368 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:23.968375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:23.968561 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:23.968598 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:09:23.968694 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 13:09:23.968822 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20541,"thread_start_us":13707,"threads_started":1}
I20260809 13:09:23.969946 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0b2854dc914cd5a4bc5a1e0db2b886" candidate_uuid: "344c262ec438447b88a371ea37661be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" is_pre_election: true
I20260809 13:09:23.970207 14375 raft_consensus.cc:2454] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344c262ec438447b88a371ea37661be9 in term 0.
I20260809 13:09:23.971252 14491 raft_consensus.cc:504] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.972292 14491 leader_election.cc:288] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [CANDIDATE]: Term 1 election: Requested vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037)
I20260809 13:09:23.972666 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0b2854dc914cd5a4bc5a1e0db2b886" candidate_uuid: "344c262ec438447b88a371ea37661be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871"
I20260809 13:09:23.972880 13938 raft_consensus.cc:3037] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.972798 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0b2854dc914cd5a4bc5a1e0db2b886" candidate_uuid: "344c262ec438447b88a371ea37661be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9"
I20260809 13:09:23.973008 14375 raft_consensus.cc:3037] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:23.975183 13938 raft_consensus.cc:2454] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344c262ec438447b88a371ea37661be9 in term 1.
I20260809 13:09:23.975647 14047 leader_election.cc:302] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:23.975906 14491 raft_consensus.cc:2781] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:23.975906 14375 raft_consensus.cc:2454] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344c262ec438447b88a371ea37661be9 in term 1.
I20260809 13:09:23.981721 14491 raft_consensus.cc:686] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 1 LEADER]: Becoming Leader. State: Replica: 344c262ec438447b88a371ea37661be9, State: Running, Role: LEADER
I20260809 13:09:23.982106 14491 consensus_queue.cc:237] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:23.985212 13669 catalog_manager.cc:5495] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 344c262ec438447b88a371ea37661be9 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "344c262ec438447b88a371ea37661be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } health_report { overall_health: UNKNOWN } } }
I20260809 13:09:24.027849 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:24.022155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.10.181.130:43025)
0809 13:09:24.022322 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:24.022351 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:24.022438 (+ 87us) server_negotiation.cc:407] Connection header received
0809 13:09:24.022623 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:24.022629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:24.022724 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:24.022884 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:24.023193 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.024291 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.025523 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.026130 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.026256 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:24.026586 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:24.027274 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:24.027412 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:24.027469 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 13:09:24.027558 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:09:24.044901 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:24.038655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.10.181.130:33261)
0809 13:09:24.039225 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:24.039241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:24.039357 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:24.039762 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:24.039798 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:24.039813 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:24.040265 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:24.040281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.041763 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.041767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:24.042840 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:24.042852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.043815 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.043819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:24.043940 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:24.043995 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:24.044430 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:24.044437 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 13:09:24.044491 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 13:09:24.044593 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":298,"threads_started":1}
I20260809 13:09:24.044906 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:24.038631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:33261 (local address 127.10.181.129:38213)
0809 13:09:24.038795 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:24.038799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:24.039328 (+ 529us) server_negotiation.cc:407] Connection header received
0809 13:09:24.039498 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:24.039504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:24.039587 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:24.039704 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:24.040494 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.041628 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.042984 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.043642 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.043798 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:24.044177 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:24.044340 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:24.044437 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:24.044552 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 13:09:24.044633 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:24.053818 14375 raft_consensus.cc:1261] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Refusing update from remote peer 344c262ec438447b88a371ea37661be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:24.054095 13938 raft_consensus.cc:1261] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Refusing update from remote peer 344c262ec438447b88a371ea37661be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:24.054782 14560 consensus_queue.cc:1035] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:24.055184 14491 consensus_queue.cc:1035] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:24.087580 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:24.081840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53557 (local address 127.10.181.130:43025)
0809 13:09:24.082027 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:24.082034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:24.082055 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:09:24.082192 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:24.082197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:24.082277 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:24.082373 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:24.082782 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.083863 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.085413 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.086092 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.086285 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:24.086805 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:24.086959 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:24.087038 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:24.087189 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:09:24.087315 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:09:24.087668 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:24.081719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43025 (local address 127.10.181.129:53557)
0809 13:09:24.081979 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:24.081997 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:24.082089 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:24.082404 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:24.082413 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:24.082426 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:24.082641 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:24.082653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.084012 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.084018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:24.085252 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:24.085272 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:24.086297 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:24.086304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:24.086523 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:24.086575 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:24.087062 (+ 487us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:24.087075 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 13:09:24.087166 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:09:24.087306 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 13:09:24.088867 14523 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 13:09:25.696888 13669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:25.697134 13669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:25.701282 13669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Sending DeleteTablet for 3 replicas of tablet 1c0b2854dc914cd5a4bc5a1e0db2b886
I20260809 13:09:25.702044 13897 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:25 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33466
I20260809 13:09:25.702097 14127 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:25 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260809 13:09:25.702148 14344 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:25 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:58942
I20260809 13:09:25.717658 14609 tablet_replica.cc:330] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: stopping tablet replica
I20260809 13:09:25.717909 14609 raft_consensus.cc:2229] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:25.718348 14609 raft_consensus.cc:2258] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:25.719429 14609 ts_tablet_manager.cc:1905] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:25.726251 14609 ts_tablet_manager.cc:1918] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:25.726410 14609 log.cc:1196] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/1c0b2854dc914cd5a4bc5a1e0db2b886
I20260809 13:09:25.726845 14609 ts_tablet_manager.cc:1939] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting consensus metadata
I20260809 13:09:25.726933 14607 tablet_replica.cc:330] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: stopping tablet replica
I20260809 13:09:25.727139 14607 raft_consensus.cc:2229] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:25.727507 14608 tablet_replica.cc:330] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: stopping tablet replica
I20260809 13:09:25.727759 14607 raft_consensus.cc:2258] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:25.727777 14608 raft_consensus.cc:2229] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:25.728127 14608 raft_consensus.cc:2258] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:25.728060 13640 catalog_manager.cc:4841] TS 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037): tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (table test-table [id=74af7cc2edf844b1845d7266c18f760c]) successfully deleted
I20260809 13:09:25.728848 14607 ts_tablet_manager.cc:1905] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:25.729149 14608 ts_tablet_manager.cc:1905] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:25.734591 14608 ts_tablet_manager.cc:1918] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:25.734730 14608 log.cc:1196] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/1c0b2854dc914cd5a4bc5a1e0db2b886
I20260809 13:09:25.735124 14608 ts_tablet_manager.cc:1939] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P b7af90a1faf045c4a746e140167f5871: Deleting consensus metadata
I20260809 13:09:25.735258 14607 ts_tablet_manager.cc:1918] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:25.735397 14607 log.cc:1196] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/1c0b2854dc914cd5a4bc5a1e0db2b886
I20260809 13:09:25.735827 14607 ts_tablet_manager.cc:1939] T 1c0b2854dc914cd5a4bc5a1e0db2b886 P 344c262ec438447b88a371ea37661be9: Deleting consensus metadata
I20260809 13:09:25.736218 13638 catalog_manager.cc:4841] TS b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213): tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (table test-table [id=74af7cc2edf844b1845d7266c18f760c]) successfully deleted
I20260809 13:09:25.736891 13638 catalog_manager.cc:4841] TS 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025): tablet 1c0b2854dc914cd5a4bc5a1e0db2b886 (table test-table [id=74af7cc2edf844b1845d7266c18f760c]) successfully deleted
Aug 09 13:09:25 dist-test-slave-42z9 krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 13:09:25 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:25.770603 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:25.757291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.10.181.190:42429)
0809 13:09:25.757929 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:25.757934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:25.758004 (+ 70us) server_negotiation.cc:407] Connection header received
0809 13:09:25.758189 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:25.758194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:25.758262 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:25.758358 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:25.759636 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:25.760429 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:25.761598 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:25.762018 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:25.762113 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:25.765858 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:25.765863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:25.765867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:25.765901 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:25.768332 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:25.768925 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:25.768928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:25.768930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:25.769050 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:25.769310 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:25.769313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:25.769315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:25.769631 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:25.769818 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:25.770086 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 13:09:25.770273 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":311,"threads_started":1}
I20260809 13:09:25.773993 13669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:25.774765 13669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:25.781778 13897 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9765d3fa4454a8b51f04a717d25df (DEFAULT_TABLE table=test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:25.782145 14127 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9765d3fa4454a8b51f04a717d25df (DEFAULT_TABLE table=test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:25.782248 13897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9765d3fa4454a8b51f04a717d25df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:25.782511 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9765d3fa4454a8b51f04a717d25df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:25.783200 14344 tablet_service.cc:1467] Processing CreateTablet for tablet d5e9765d3fa4454a8b51f04a717d25df (DEFAULT_TABLE table=test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:25.783643 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e9765d3fa4454a8b51f04a717d25df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:25.797987 14622 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Bootstrap starting.
I20260809 13:09:25.798666 14625 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Bootstrap starting.
I20260809 13:09:25.800151 14622 tablet_bootstrap.cc:654] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:25.800555 14625 tablet_bootstrap.cc:654] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:25.802240 14623 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Bootstrap starting.
I20260809 13:09:25.803840 14623 tablet_bootstrap.cc:654] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:25.805159 14625 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: No bootstrap required, opened a new log
I20260809 13:09:25.805295 14625 ts_tablet_manager.cc:1397] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20260809 13:09:25.806165 14625 raft_consensus.cc:348] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.806412 14625 raft_consensus.cc:374] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:25.806506 14625 raft_consensus.cc:729] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Initialized, Role: FOLLOWER
I20260809 13:09:25.806538 14623 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: No bootstrap required, opened a new log
I20260809 13:09:25.806856 14622 tablet_bootstrap.cc:492] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: No bootstrap required, opened a new log
I20260809 13:09:25.806779 14625 consensus_queue.cc:260] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.807113 14622 ts_tablet_manager.cc:1397] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 13:09:25.806715 14623 ts_tablet_manager.cc:1397] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 13:09:25.808089 14622 raft_consensus.cc:348] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.808408 14622 raft_consensus.cc:374] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:25.808547 14622 raft_consensus.cc:729] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Initialized, Role: FOLLOWER
I20260809 13:09:25.808419 14623 raft_consensus.cc:348] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.808657 14623 raft_consensus.cc:374] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:25.808750 14623 raft_consensus.cc:729] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344c262ec438447b88a371ea37661be9, State: Initialized, Role: FOLLOWER
I20260809 13:09:25.808969 14623 consensus_queue.cc:260] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.809536 14623 ts_tablet_manager.cc:1428] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 13:09:25.808964 14622 consensus_queue.cc:260] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.812784 14622 ts_tablet_manager.cc:1428] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 13:09:25.815443 14625 ts_tablet_manager.cc:1428] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent starting tablet: real 0.010s user 0.002s sys 0.008s
I20260809 13:09:25.996387 14629 raft_consensus.cc:482] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:25.996654 14629 raft_consensus.cc:504] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:25.998262 14629 leader_election.cc:288] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:26.005422 14640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:25.998503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.10.181.131:44983)
0809 13:09:25.999096 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:25.999136 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:25.999277 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:25.999883 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:25.999888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:25.999924 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:26.000264 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.000274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.001627 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.001632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.003225 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.003242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.004555 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.004562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.004761 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.004798 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:09:26.004907 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 13:09:26.005048 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"spinlock_wait_cycles":51456,"thread_start_us":306,"threads_started":1}
I20260809 13:09:26.005470 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:25.998600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:44983 (local address 127.10.181.129:38213)
0809 13:09:25.999226 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:25.999232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:25.999259 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:09:25.999476 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:25.999481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:25.999706 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:26.000391 (+ 685us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:26.000572 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.001498 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.003405 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.004357 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.004482 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.004538 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:26.004944 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 13:09:26.005090 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":351,"threads_started":1}
I20260809 13:09:26.005869 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:25.999237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43025 (local address 127.10.181.131:53601)
0809 13:09:25.999925 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:25.999943 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:26.000126 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:26.000502 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:26.000507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:26.000530 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:26.000859 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.000868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.002161 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.002165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.003662 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.003677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.005042 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.005049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.005248 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.005287 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 13:09:26.005390 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 13:09:26.005545 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"spinlock_wait_cycles":18688,"thread_start_us":374,"threads_started":1}
I20260809 13:09:26.005927 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:25.999395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:53601 (local address 127.10.181.130:43025)
0809 13:09:26.000241 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:26.000247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:26.000266 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:09:26.000322 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:26.000325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:26.000379 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:26.000462 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:26.001040 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.002033 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.003837 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.004845 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.005018 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.005086 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:26.005432 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 13:09:26.005588 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":549,"threads_started":1}
I20260809 13:09:26.006381 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9765d3fa4454a8b51f04a717d25df" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871" is_pre_election: true
I20260809 13:09:26.006500 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9765d3fa4454a8b51f04a717d25df" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9" is_pre_election: true
I20260809 13:09:26.006683 13937 raft_consensus.cc:2454] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 0.
I20260809 13:09:26.006743 14159 raft_consensus.cc:2454] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 0.
I20260809 13:09:26.007236 14265 leader_election.cc:302] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 344c262ec438447b88a371ea37661be9, 4f80fb4ab8cf4eb384e8e56a749975a9; no voters:
I20260809 13:09:26.007611 14629 raft_consensus.cc:2781] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:26.007751 14629 raft_consensus.cc:482] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:26.007862 14629 raft_consensus.cc:3037] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.010097 14629 raft_consensus.cc:504] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.010715 14629 leader_election.cc:288] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 election: Requested vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:26.011188 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9765d3fa4454a8b51f04a717d25df" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871"
I20260809 13:09:26.011264 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e9765d3fa4454a8b51f04a717d25df" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9"
I20260809 13:09:26.011384 13937 raft_consensus.cc:3037] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.011492 14159 raft_consensus.cc:3037] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.014182 13937 raft_consensus.cc:2454] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 1.
I20260809 13:09:26.014302 14159 raft_consensus.cc:2454] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 1.
I20260809 13:09:26.014708 14265 leader_election.cc:302] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f80fb4ab8cf4eb384e8e56a749975a9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:26.015051 14629 raft_consensus.cc:2781] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:26.015750 14629 raft_consensus.cc:686] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 LEADER]: Becoming Leader. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Running, Role: LEADER
I20260809 13:09:26.016189 14629 consensus_queue.cc:237] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.019580 13667 catalog_manager.cc:5495] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } health_report { overall_health: HEALTHY } } }
I20260809 13:09:26.044196 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:26.038695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.10.181.131:36037)
0809 13:09:26.039459 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:26.039464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:26.039485 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:09:26.039573 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:26.039576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:26.039669 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:26.039821 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:26.040136 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.040917 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.042187 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.042594 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.042692 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.043073 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:26.043694 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:26.043809 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:26.043854 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 13:09:26.043919 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":499,"threads_started":1}
I20260809 13:09:26.058979 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:26.054023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:38609 (local address 127.10.181.129:38213)
0809 13:09:26.054266 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:26.054272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:26.054587 (+ 315us) server_negotiation.cc:407] Connection header received
0809 13:09:26.054746 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:26.054751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:26.054839 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:26.054941 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:26.055441 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.056194 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.057543 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.057957 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.058044 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.058455 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:26.058558 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:26.058635 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:26.058708 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 13:09:26.058771 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:09:26.059108 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:26.053851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:38213 (local address 127.10.181.131:38609)
0809 13:09:26.054500 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:26.054515 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:26.054637 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:26.054962 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:26.054965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:26.054976 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:26.055318 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.055327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.056315 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.056319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.057406 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.057419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.058110 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.058115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.058240 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.058310 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:26.058629 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:26.058636 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 13:09:26.058687 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 13:09:26.058784 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":331,"threads_started":1}
I20260809 13:09:26.068081 14159 raft_consensus.cc:1261] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Refusing update from remote peer 4f80fb4ab8cf4eb384e8e56a749975a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:26.068202 13937 raft_consensus.cc:1261] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Refusing update from remote peer 4f80fb4ab8cf4eb384e8e56a749975a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:26.068819 14644 consensus_queue.cc:1035] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:26.069164 14629 consensus_queue.cc:1035] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:26.096971 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:26.091557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:56087 (local address 127.10.181.131:36037)
0809 13:09:26.091712 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:26.091717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:26.092278 (+ 561us) server_negotiation.cc:407] Connection header received
0809 13:09:26.092451 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:26.092456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:26.092526 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:26.092598 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:26.093079 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.093913 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.095473 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.095976 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.096072 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.096474 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:26.096571 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:26.096630 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:26.096706 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 13:09:26.096766 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 13:09:26.097047 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:26.091542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:36037 (local address 127.10.181.129:56087)
0809 13:09:26.092181 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:26.092197 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:26.092326 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:26.092634 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:26.092638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:26.092651 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:09:26.092941 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.092949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.094056 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.094061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.095340 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:26.095357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:26.096170 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:26.096175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:26.096305 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:26.096344 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:09:26.096633 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:09:26.096639 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 13:09:26.096692 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 13:09:26.096787 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":416,"threads_started":1}
I20260809 13:09:26.127810 13669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:26.128093 13669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:26.131354 13669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Sending DeleteTablet for 3 replicas of tablet d5e9765d3fa4454a8b51f04a717d25df
I20260809 13:09:26.132146 13897 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9765d3fa4454a8b51f04a717d25df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:26 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33466
I20260809 13:09:26.132177 14127 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9765d3fa4454a8b51f04a717d25df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:26 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260809 13:09:26.132187 14344 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e9765d3fa4454a8b51f04a717d25df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:26 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:58942
I20260809 13:09:26.132475 14609 tablet_replica.cc:330] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: stopping tablet replica
I20260809 13:09:26.132501 14608 tablet_replica.cc:330] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: stopping tablet replica
I20260809 13:09:26.132691 14609 raft_consensus.cc:2229] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:26.132736 14608 raft_consensus.cc:2229] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:26.132787 14607 tablet_replica.cc:330] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: stopping tablet replica
I20260809 13:09:26.132938 14608 raft_consensus.cc:2258] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:26.132970 14607 raft_consensus.cc:2229] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:26.133169 14607 raft_consensus.cc:2258] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:26.133217 14609 raft_consensus.cc:2258] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:26.134218 14608 ts_tablet_manager.cc:1905] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:26.134452 14607 ts_tablet_manager.cc:1905] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:26.135751 14609 ts_tablet_manager.cc:1905] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:26.140870 14607 ts_tablet_manager.cc:1918] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:26.140870 14608 ts_tablet_manager.cc:1918] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:26.141014 14608 log.cc:1196] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/d5e9765d3fa4454a8b51f04a717d25df
I20260809 13:09:26.141014 14607 log.cc:1196] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/d5e9765d3fa4454a8b51f04a717d25df
I20260809 13:09:26.141427 14608 ts_tablet_manager.cc:1939] T d5e9765d3fa4454a8b51f04a717d25df P b7af90a1faf045c4a746e140167f5871: Deleting consensus metadata
I20260809 13:09:26.141969 13669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 13:09:26.142503 13638 catalog_manager.cc:4841] TS b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213): tablet d5e9765d3fa4454a8b51f04a717d25df (table test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]) successfully deleted
W20260809 13:09:26.142792 13669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:26.142823 14607 ts_tablet_manager.cc:1939] T d5e9765d3fa4454a8b51f04a717d25df P 344c262ec438447b88a371ea37661be9: Deleting consensus metadata
I20260809 13:09:26.142954 14609 ts_tablet_manager.cc:1918] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:09:26.143136 14609 log.cc:1196] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/d5e9765d3fa4454a8b51f04a717d25df
I20260809 13:09:26.143589 14609 ts_tablet_manager.cc:1939] T d5e9765d3fa4454a8b51f04a717d25df P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting consensus metadata
I20260809 13:09:26.143893 13638 catalog_manager.cc:4841] TS 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025): tablet d5e9765d3fa4454a8b51f04a717d25df (table test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]) successfully deleted
I20260809 13:09:26.144704 13640 catalog_manager.cc:4841] TS 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037): tablet d5e9765d3fa4454a8b51f04a717d25df (table test-table [id=d779599c8cd14d9ab5b32d236faa8c4c]) successfully deleted
I20260809 13:09:26.149873 14127 tablet_service.cc:1467] Processing CreateTablet for tablet 71e034c43f2240038bd479ad44cb3908 (DEFAULT_TABLE table=test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:26.150075 14344 tablet_service.cc:1467] Processing CreateTablet for tablet 71e034c43f2240038bd479ad44cb3908 (DEFAULT_TABLE table=test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:26.150314 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e034c43f2240038bd479ad44cb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:26.150296 13897 tablet_service.cc:1467] Processing CreateTablet for tablet 71e034c43f2240038bd479ad44cb3908 (DEFAULT_TABLE table=test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:26.150486 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e034c43f2240038bd479ad44cb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:26.150701 13897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e034c43f2240038bd479ad44cb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:26.156648 14623 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Bootstrap starting.
I20260809 13:09:26.156939 14622 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Bootstrap starting.
I20260809 13:09:26.156970 14625 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Bootstrap starting.
I20260809 13:09:26.158365 14623 tablet_bootstrap.cc:654] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:26.158375 14622 tablet_bootstrap.cc:654] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:26.158618 14625 tablet_bootstrap.cc:654] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:26.161345 14622 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: No bootstrap required, opened a new log
I20260809 13:09:26.161558 14622 ts_tablet_manager.cc:1397] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 13:09:26.162361 14622 raft_consensus.cc:348] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.162636 14622 raft_consensus.cc:374] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:26.162724 14622 raft_consensus.cc:729] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7af90a1faf045c4a746e140167f5871, State: Initialized, Role: FOLLOWER
I20260809 13:09:26.163466 14622 consensus_queue.cc:260] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.164031 14622 ts_tablet_manager.cc:1428] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 13:09:26.167634 14625 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: No bootstrap required, opened a new log
I20260809 13:09:26.167869 14625 ts_tablet_manager.cc:1397] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260809 13:09:26.168773 14625 raft_consensus.cc:348] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.169042 14623 tablet_bootstrap.cc:492] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: No bootstrap required, opened a new log
I20260809 13:09:26.169067 14625 raft_consensus.cc:374] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:26.169158 14625 raft_consensus.cc:729] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Initialized, Role: FOLLOWER
I20260809 13:09:26.169217 14623 ts_tablet_manager.cc:1397] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260809 13:09:26.169480 14625 consensus_queue.cc:260] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.170027 14623 raft_consensus.cc:348] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.170291 14623 raft_consensus.cc:374] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:26.170387 14623 raft_consensus.cc:729] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344c262ec438447b88a371ea37661be9, State: Initialized, Role: FOLLOWER
I20260809 13:09:26.170676 14623 consensus_queue.cc:260] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.171211 14623 ts_tablet_manager.cc:1428] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 13:09:26.172139 14625 ts_tablet_manager.cc:1428] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 13:09:26.253921 14661 raft_consensus.cc:482] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:26.254191 14661 raft_consensus.cc:504] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.255152 14661 leader_election.cc:288] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:26.255879 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e034c43f2240038bd479ad44cb3908" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9" is_pre_election: true
I20260809 13:09:26.255892 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e034c43f2240038bd479ad44cb3908" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871" is_pre_election: true
I20260809 13:09:26.256181 13937 raft_consensus.cc:2454] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 0.
I20260809 13:09:26.256177 14159 raft_consensus.cc:2454] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 0.
I20260809 13:09:26.256700 14265 leader_election.cc:302] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f80fb4ab8cf4eb384e8e56a749975a9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:26.257005 14661 raft_consensus.cc:2781] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:26.257131 14661 raft_consensus.cc:482] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:26.257234 14661 raft_consensus.cc:3037] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.259635 14661 raft_consensus.cc:504] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.260171 14661 leader_election.cc:288] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 election: Requested vote from peers b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213), 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025)
I20260809 13:09:26.260758 14159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e034c43f2240038bd479ad44cb3908" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "344c262ec438447b88a371ea37661be9"
I20260809 13:09:26.260758 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e034c43f2240038bd479ad44cb3908" candidate_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7af90a1faf045c4a746e140167f5871"
I20260809 13:09:26.260982 14159 raft_consensus.cc:3037] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.260982 13937 raft_consensus.cc:3037] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:26.263839 14159 raft_consensus.cc:2454] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 1.
I20260809 13:09:26.263839 13937 raft_consensus.cc:2454] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f80fb4ab8cf4eb384e8e56a749975a9 in term 1.
I20260809 13:09:26.264252 14265 leader_election.cc:302] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f80fb4ab8cf4eb384e8e56a749975a9, b7af90a1faf045c4a746e140167f5871; no voters:
I20260809 13:09:26.264532 14661 raft_consensus.cc:2781] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:26.264700 14661 raft_consensus.cc:686] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 LEADER]: Becoming Leader. State: Replica: 4f80fb4ab8cf4eb384e8e56a749975a9, State: Running, Role: LEADER
I20260809 13:09:26.264977 14661 consensus_queue.cc:237] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } }
I20260809 13:09:26.268253 13667 catalog_manager.cc:5495] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f80fb4ab8cf4eb384e8e56a749975a9" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 36037 } health_report { overall_health: HEALTHY } } }
I20260809 13:09:26.309406 13937 raft_consensus.cc:1261] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Refusing update from remote peer 4f80fb4ab8cf4eb384e8e56a749975a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:26.309480 14156 raft_consensus.cc:1261] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Refusing update from remote peer 4f80fb4ab8cf4eb384e8e56a749975a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:26.310225 14629 consensus_queue.cc:1035] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7af90a1faf045c4a746e140167f5871" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 38213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:26.310686 14644 consensus_queue.cc:1035] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "344c262ec438447b88a371ea37661be9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:26.317286 14582 mvcc.cc:205] Tried to move back new op lower bound from 7316606837997522944 to 7316606837824684032. Current Snapshot: MvccSnapshot[applied={T|T < 7316606837997522944}]
W20260809 13:09:26.340153 14582 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 13:09:27.930609 13669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:27.930835 13669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:27.934518 13669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd90a13aaeca418e872ce8799fb8455d: Sending DeleteTablet for 3 replicas of tablet 71e034c43f2240038bd479ad44cb3908
I20260809 13:09:27.935168 13897 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e034c43f2240038bd479ad44cb3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:27 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33466
I20260809 13:09:27.935298 14344 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e034c43f2240038bd479ad44cb3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:27 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:58942
I20260809 13:09:27.935321 14127 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e034c43f2240038bd479ad44cb3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:27 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260809 13:09:27.935911 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13792
I20260809 13:09:27.948678 14712 tablet_replica.cc:330] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: stopping tablet replica
I20260809 13:09:27.948896 14712 raft_consensus.cc:2229] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:27.949259 14712 raft_consensus.cc:2258] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:27.949836 14707 tablet_replica.cc:330] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: stopping tablet replica
I20260809 13:09:27.950183 14707 raft_consensus.cc:2229] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:27.950536 14707 raft_consensus.cc:2258] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:27.951081 14709 tablet_replica.cc:330] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: stopping tablet replica
I20260809 13:09:27.951267 14709 raft_consensus.cc:2229] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:27.951328 14712 ts_tablet_manager.cc:1905] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:27.951517 14709 raft_consensus.cc:2258] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:27.952276 14707 ts_tablet_manager.cc:1905] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:27.953434 14709 ts_tablet_manager.cc:1905] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:27.957821 14712 ts_tablet_manager.cc:1918] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:27.957974 14712 log.cc:1196] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/71e034c43f2240038bd479ad44cb3908
I20260809 13:09:27.958480 14712 ts_tablet_manager.cc:1939] T 71e034c43f2240038bd479ad44cb3908 P 4f80fb4ab8cf4eb384e8e56a749975a9: Deleting consensus metadata
I20260809 13:09:27.959264 14707 ts_tablet_manager.cc:1918] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:27.959410 14707 log.cc:1196] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/71e034c43f2240038bd479ad44cb3908
I20260809 13:09:27.959612 14709 ts_tablet_manager.cc:1918] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 13:09:27.959604 13640 catalog_manager.cc:4841] TS 4f80fb4ab8cf4eb384e8e56a749975a9 (127.10.181.131:36037): tablet 71e034c43f2240038bd479ad44cb3908 (table test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]) successfully deleted
I20260809 13:09:27.959724 14709 log.cc:1196] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/71e034c43f2240038bd479ad44cb3908
I20260809 13:09:27.959858 14707 ts_tablet_manager.cc:1939] T 71e034c43f2240038bd479ad44cb3908 P b7af90a1faf045c4a746e140167f5871: Deleting consensus metadata
I20260809 13:09:27.960110 14709 ts_tablet_manager.cc:1939] T 71e034c43f2240038bd479ad44cb3908 P 344c262ec438447b88a371ea37661be9: Deleting consensus metadata
I20260809 13:09:27.960783 13638 catalog_manager.cc:4841] TS b7af90a1faf045c4a746e140167f5871 (127.10.181.129:38213): tablet 71e034c43f2240038bd479ad44cb3908 (table test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]) successfully deleted
I20260809 13:09:27.961033 13638 catalog_manager.cc:4841] TS 344c262ec438447b88a371ea37661be9 (127.10.181.130:43025): tablet 71e034c43f2240038bd479ad44cb3908 (table test-table [id=099224d3ae7d4a37bfd3bdf367ab14c1]) successfully deleted
Aug 09 13:09:27 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:27.970256 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:27.938727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.10.181.129:38213)
0809 13:09:27.950893 (+ 12166us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:27.950898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:27.950922 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:09:27.950992 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:27.950997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:27.951083 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:27.951190 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:27.952893 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:27.954057 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:27.955922 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:27.956590 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:27.956740 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:27.964782 (+ 8042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:27.964800 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:27.964818 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:27.964851 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:27.967445 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:27.968085 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:27.968093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:27.968096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:27.968263 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:27.968635 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:27.968639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:27.968641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:27.969017 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:27.969311 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:27.969668 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 13:09:27.969893 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12053,"thread_start_us":11862,"threads_started":1}
W20260809 13:09:27.970901 14000 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33496
W20260809 13:09:27.971473 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:27.971678 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13792
I20260809 13:09:28.007530 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14014
Aug 09 13:09:28 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:28.022742 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:28.008292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.10.181.130:43025)
0809 13:09:28.008817 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:28.008821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:28.008838 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:09:28.008895 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:28.008899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:28.008979 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:28.009070 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:28.010362 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:28.011143 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:28.012275 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:28.012749 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:28.012843 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:28.016702 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:28.016720 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:28.016742 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:28.016784 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:28.019946 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:28.020722 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:28.020732 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:28.020736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:28.020866 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:28.021299 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:28.021305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:28.021308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:28.021796 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:28.022042 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:28.022272 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 13:09:28.022436 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":274,"threads_started":1}
W20260809 13:09:28.023381 14227 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60982
W20260809 13:09:28.024001 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14014: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:28.024160 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14014
I20260809 13:09:28.054718 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14240
Aug 09 13:09:28 dist-test-slave-42z9 krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280965, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:28.068651 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:28.055477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.10.181.131:36037)
0809 13:09:28.055998 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:28.056003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:28.056020 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:09:28.056077 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:28.056081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:28.056161 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:28.056253 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:28.057497 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:28.058207 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:28.059342 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:28.059755 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:28.059845 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:28.063756 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:28.063771 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:28.063791 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:28.063825 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:28.066410 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:28.066977 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:28.066984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:28.066986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:28.067110 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:28.067364 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:28.067367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:28.067369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:28.067718 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:28.067936 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:28.068147 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 13:09:28.068315 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":252,"threads_started":1}
W20260809 13:09:28.069258 14445 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58952
W20260809 13:09:28.069876 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14240: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:28.070055 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14240
I20260809 13:09:28.105819 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13611
I20260809 13:09:28.106992 13743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:28.222806 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13611
2026-08-09T13:09:28Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8399 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:28 dist-test-slave-42z9 krb5kdc[14726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:28 dist-test-slave-42z9 krb5kdc[14726](info): set up 2 sockets
Aug 09 13:09:28 dist-test-slave-42z9 krb5kdc[14726](info): commencing operation
krb5kdc: starting...
W20260809 13:09:29.411288 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:29 dist-test-slave-42z9 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:29.603955 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:38035
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:38035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:29.925202 14745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:29.925508 14745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:29.925650 14745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:29.932581 14745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:29.932683 14745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:29.932762 14745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:29.932842 14745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:29.932904 14745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:29.942193 14745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:38035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:38035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:29.943356 14745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:29.944603 14745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:29.955354 14752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:29.955438 14754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:29.955399 14756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:29.956452 14745 server_base.cc:1034] running on GCE node
I20260809 13:09:29.957116 14745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:29.958374 14745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:29.959575 14745 hybrid_clock.cc:648] HybridClock initialized: now 1786280969959526 us; error 32 us; skew 500 ppm
I20260809 13:09:29.959857 14745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:29 dist-test-slave-42z9 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280969, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:29.964824 14745 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:29.966255 14745 webserver.cc:460] Webserver started at http://127.10.181.190:45259/ using document root <none> and password file <none>
I20260809 13:09:29.966861 14745 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:29.966956 14745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:29.967202 14745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:29.970009 14745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "ccde65e6227c470d8ad2eed0d54408f2"
format_stamp: "Formatted at 2026-08-09 13:09:29 on dist-test-slave-42z9"
server_key: "c303a96a44e05fe4d90eb29012925b7c"
server_key_iv: "0568a5a17ca77b03d5ac259671d491bb"
server_key_version: "encryptionkey@0"
I20260809 13:09:29.970697 14745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "ccde65e6227c470d8ad2eed0d54408f2"
format_stamp: "Formatted at 2026-08-09 13:09:29 on dist-test-slave-42z9"
server_key: "c303a96a44e05fe4d90eb29012925b7c"
server_key_iv: "0568a5a17ca77b03d5ac259671d491bb"
server_key_version: "encryptionkey@0"
I20260809 13:09:29.975454 14745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:09:29.978679 14767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:29.979894 14745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:29.980036 14745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "ccde65e6227c470d8ad2eed0d54408f2"
format_stamp: "Formatted at 2026-08-09 13:09:29 on dist-test-slave-42z9"
server_key: "c303a96a44e05fe4d90eb29012925b7c"
server_key_iv: "0568a5a17ca77b03d5ac259671d491bb"
server_key_version: "encryptionkey@0"
I20260809 13:09:29.980187 14745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:29.999545 14745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:30.003043 14745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:30.003254 14745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:30.020185 14745 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:38035
I20260809 13:09:30.020217 14855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:38035 every 8 connection(s)
I20260809 13:09:30.021466 14745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:30.025198 14858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:30.028903 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14745
I20260809 13:09:30.029147 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:30.029959 10966 external_mini_cluster.cc:1428] Setting key e92983406eca75cef32498ba38b87156
I20260809 13:09:30.037858 14858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2: Bootstrap starting.
I20260809 13:09:30.040943 14858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:30.041965 14858 log.cc:824] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:09:30 dist-test-slave-42z9 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:30.044814 14858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2: No bootstrap required, opened a new log
I20260809 13:09:30.052325 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:30.032101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.10.181.190:38035)
0809 13:09:30.032841 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:30.032852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:30.032891 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:09:30.034593 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:30.034615 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:30.034983 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:30.035327 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:30.036752 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.038117 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.039334 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.040006 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.040168 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:30.044078 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:30.044090 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:30.044108 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:30.044156 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:30.048092 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:30.048728 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:30.048737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:30.048745 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:30.048947 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:30.049289 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:30.049295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:30.049298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:30.049973 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:30.050255 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:30.050627 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 13:09:30.051037 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":315,"threads_started":1}
I20260809 13:09:30.055352 14858 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } }
I20260809 13:09:30.055774 14858 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:30.055878 14858 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccde65e6227c470d8ad2eed0d54408f2, State: Initialized, Role: FOLLOWER
I20260809 13:09:30.056617 14858 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } }
I20260809 13:09:30.056900 14858 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:30.057044 14858 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:30.057243 14858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:30.060235 14858 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } }
I20260809 13:09:30.060695 14858 leader_election.cc:302] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccde65e6227c470d8ad2eed0d54408f2; no voters:
I20260809 13:09:30.061967 14858 leader_election.cc:288] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:30.062083 14865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:30.062743 14865 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [term 1 LEADER]: Becoming Leader. State: Replica: ccde65e6227c470d8ad2eed0d54408f2, State: Running, Role: LEADER
I20260809 13:09:30.063288 14865 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } }
I20260809 13:09:30.063923 14858 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:30.066033 14866 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccde65e6227c470d8ad2eed0d54408f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } } }
I20260809 13:09:30.066406 14866 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:30.066928 14867 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccde65e6227c470d8ad2eed0d54408f2. Latest consensus state: current_term: 1 leader_uuid: "ccde65e6227c470d8ad2eed0d54408f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccde65e6227c470d8ad2eed0d54408f2" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38035 } } }
I20260809 13:09:30.067231 14867 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:30.068296 14872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:30.074239 14872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:30.084389 14872 catalog_manager.cc:1347] Generated new cluster ID: 3ee0fa6735a741f3bb16ee23c14adbd1
I20260809 13:09:30.084515 14872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:30.110005 14872 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:30.111910 14872 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:30.119935 14872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccde65e6227c470d8ad2eed0d54408f2: Generated new TSK 0
I20260809 13:09:30.120927 14872 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:30.176544 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:30.502266 14900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:30.502552 14900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:30.502699 14900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:30.509900 14900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:30.510018 14900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:30.510179 14900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:30.519649 14900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:30.520910 14900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:30.522243 14900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:30.534488 14909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:30.534530 14910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:30.534622 14914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:30.535501 14900 server_base.cc:1034] running on GCE node
I20260809 13:09:30.536088 14900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:30.536959 14900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:30.538149 14900 hybrid_clock.cc:648] HybridClock initialized: now 1786280970538106 us; error 31 us; skew 500 ppm
I20260809 13:09:30.538443 14900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:30 dist-test-slave-42z9 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280970, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:30.541345 14900 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:30.542749 14900 webserver.cc:460] Webserver started at http://127.10.181.129:40717/ using document root <none> and password file <none>
I20260809 13:09:30.543412 14900 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:30.543522 14900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:30.543804 14900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:30.546595 14900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "bea78002052045aa86a8cea3886a5d26"
format_stamp: "Formatted at 2026-08-09 13:09:30 on dist-test-slave-42z9"
server_key: "b3d3a89d35c771c44f0facb3f9095d06"
server_key_iv: "60f65a01f5bb25467d2be5b42f4cbf7f"
server_key_version: "encryptionkey@0"
I20260809 13:09:30.547339 14900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "bea78002052045aa86a8cea3886a5d26"
format_stamp: "Formatted at 2026-08-09 13:09:30 on dist-test-slave-42z9"
server_key: "b3d3a89d35c771c44f0facb3f9095d06"
server_key_iv: "60f65a01f5bb25467d2be5b42f4cbf7f"
server_key_version: "encryptionkey@0"
I20260809 13:09:30.552304 14900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:30.555536 14922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:30.556712 14900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 13:09:30.556860 14900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "bea78002052045aa86a8cea3886a5d26"
format_stamp: "Formatted at 2026-08-09 13:09:30 on dist-test-slave-42z9"
server_key: "b3d3a89d35c771c44f0facb3f9095d06"
server_key_iv: "60f65a01f5bb25467d2be5b42f4cbf7f"
server_key_version: "encryptionkey@0"
I20260809 13:09:30.557026 14900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:30.580425 14900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:30.584205 14900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:30.584475 14900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:30.585369 14900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:30.586647 14900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:30.586724 14900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:30.586839 14900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:30.586891 14900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:30.619074 14900 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:39471
I20260809 13:09:30.619115 15102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:39471 every 8 connection(s)
I20260809 13:09:30.620318 14900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:30.626220 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14900
I20260809 13:09:30.626458 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:30.627245 10966 external_mini_cluster.cc:1428] Setting key 99f982b71fed5bee65258699d323772c
Aug 09 13:09:30 dist-test-slave-42z9 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280970, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:30.640221 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:30.622766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:41003 (local address 127.10.181.190:38035)
0809 13:09:30.623197 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:30.623201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:30.624185 (+ 984us) server_negotiation.cc:407] Connection header received
0809 13:09:30.626098 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:30.626102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:30.626167 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:30.626261 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:30.628181 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.629035 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.630609 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.631200 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.631333 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:30.634507 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:30.634513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:30.634518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:30.634563 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:30.637293 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:30.638136 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:30.638140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:30.638142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:30.638256 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:30.638702 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:30.638705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:30.638707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:30.638944 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:30.639050 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:30.639830 (+ 780us) server_negotiation.cc:299] Negotiation successful
0809 13:09:30.639984 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":203,"threads_started":1}
I20260809 13:09:30.641566 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:30.623284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:38035 (local address 127.10.181.129:41003)
0809 13:09:30.624009 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:30.624047 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:30.625789 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:30.626540 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:30.626552 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:30.627281 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:30.628025 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:30.628046 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.629193 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.629199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:30.630452 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:30.630466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:30.631386 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:30.631391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:30.631575 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:30.632189 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:30.632211 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:30.634296 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:30.637422 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:30.637431 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:30.637462 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:30.638031 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:30.638351 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:30.638356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:30.638360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:30.638600 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:30.639073 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:30.639080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:30.639460 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 13:09:30.639751 (+ 291us) client_negotiation.cc:240] Negotiation successful
0809 13:09:30.640215 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":290,"threads_started":1}
I20260809 13:09:30.643020 15103 heartbeater.cc:347] Connected to a master server at 127.10.181.190:38035
I20260809 13:09:30.643332 15103 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:30.644109 15103 heartbeater.cc:511] Master 127.10.181.190:38035 requested a full tablet report, sending...
I20260809 13:09:30.646514 14796 ts_manager.cc:194] Registered new tserver with Master: bea78002052045aa86a8cea3886a5d26 (127.10.181.129:39471)
I20260809 13:09:30.648540 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:41003
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:30.676218 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:30.996739 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:30.997011 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:30.997107 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:31.004408 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:31.004511 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:31.004630 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:31.014238 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:31.015482 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:31.016729 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:31.029567 15119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:31.030432 15120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:31.031210 15110 server_base.cc:1034] running on GCE node
W20260809 13:09:31.031453 15122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:31.032155 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:31.033247 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:31.034485 15110 hybrid_clock.cc:648] HybridClock initialized: now 1786280971034413 us; error 58 us; skew 500 ppm
I20260809 13:09:31.034845 15110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280971, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:31.038546 15110 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:31.040058 15110 webserver.cc:460] Webserver started at http://127.10.181.130:36229/ using document root <none> and password file <none>
I20260809 13:09:31.040692 15110 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:31.040792 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:31.041059 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:31.043896 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "fdccba111cc44124baa4a997c363f5aa"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "f077f7fe9f42bb4943d786f61e5f0b95"
server_key_iv: "913558db2287b81d7c13bc8ebe46566e"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.044605 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "fdccba111cc44124baa4a997c363f5aa"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "f077f7fe9f42bb4943d786f61e5f0b95"
server_key_iv: "913558db2287b81d7c13bc8ebe46566e"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.049638 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 13:09:31.052913 15133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.054188 15110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:31.054339 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "fdccba111cc44124baa4a997c363f5aa"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "f077f7fe9f42bb4943d786f61e5f0b95"
server_key_iv: "913558db2287b81d7c13bc8ebe46566e"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.054523 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:31.065317 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:31.068815 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:31.069059 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:31.070045 15110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:31.071296 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:31.071369 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.071481 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:31.071532 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.101804 15110 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:35853
I20260809 13:09:31.101862 15307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:35853 every 8 connection(s)
I20260809 13:09:31.103065 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:31.112963 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 15110
I20260809 13:09:31.113175 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:31.113729 10966 external_mini_cluster.cc:1428] Setting key da5dddd4b568916369fdacdc347521bf
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280971, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:31.123070 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:31.105504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:36655 (local address 127.10.181.190:38035)
0809 13:09:31.105689 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:31.105694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:31.106818 (+ 1124us) server_negotiation.cc:407] Connection header received
0809 13:09:31.108627 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:31.108634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:31.108710 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:31.108797 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:31.110359 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.111079 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.112623 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.113226 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.113358 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:31.117334 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:31.117339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:31.117343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:31.117375 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:31.119818 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:31.120678 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:31.120682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:31.120684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:31.120818 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:31.121258 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:31.121261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:31.121263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:31.121557 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:31.121682 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:31.122652 (+ 970us) server_negotiation.cc:299] Negotiation successful
0809 13:09:31.122827 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:09:31.124322 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:31.105957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:38035 (local address 127.10.181.130:36655)
0809 13:09:31.106654 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:31.106696 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:31.108335 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:31.109036 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:31.109044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:31.109628 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:31.110188 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:31.110206 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.111232 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.111236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:31.112444 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:31.112457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.113468 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.113477 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:31.113694 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:31.114366 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:31.114396 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:31.117118 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:31.119961 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:31.119969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:31.119986 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:31.120535 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:31.120919 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:31.120924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:31.120927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:31.121162 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:31.121687 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:31.121713 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:31.122167 (+ 454us) client_negotiation.cc:769] Sending connection context
0809 13:09:31.122549 (+ 382us) client_negotiation.cc:240] Negotiation successful
0809 13:09:31.123055 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":278,"threads_started":1}
I20260809 13:09:31.125734 15311 heartbeater.cc:347] Connected to a master server at 127.10.181.190:38035
I20260809 13:09:31.126052 15311 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:31.126729 15311 heartbeater.cc:511] Master 127.10.181.190:38035 requested a full tablet report, sending...
I20260809 13:09:31.128185 14796 ts_manager.cc:194] Registered new tserver with Master: fdccba111cc44124baa4a997c363f5aa (127.10.181.130:35853)
I20260809 13:09:31.128959 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:36655
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:31.162792 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:31.482378 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:31.482653 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:31.482795 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:31.489948 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:31.490058 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:31.490213 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:31.499446 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:31.500627 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:31.501956 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:31.514667 15323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:31.514714 15325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:31.514743 15335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:31.515730 15317 server_base.cc:1034] running on GCE node
I20260809 13:09:31.516244 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:31.517048 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:31.518247 15317 hybrid_clock.cc:648] HybridClock initialized: now 1786280971518189 us; error 39 us; skew 500 ppm
I20260809 13:09:31.518532 15317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[14726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280971, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:31.521474 15317 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:31.522795 15317 webserver.cc:460] Webserver started at http://127.10.181.131:46511/ using document root <none> and password file <none>
I20260809 13:09:31.523389 15317 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:31.523485 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:31.523742 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:31.526466 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "30eaae0500e6452fa15c627cd8bbd41a"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "e9fb69e2b4265127f919b9de9e173628"
server_key_iv: "59fe09d1a1c4647de0c2983cc199c0ea"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.527174 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "30eaae0500e6452fa15c627cd8bbd41a"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "e9fb69e2b4265127f919b9de9e173628"
server_key_iv: "59fe09d1a1c4647de0c2983cc199c0ea"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.531966 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:09:31.535169 15346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.536358 15317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 13:09:31.536494 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "30eaae0500e6452fa15c627cd8bbd41a"
format_stamp: "Formatted at 2026-08-09 13:09:31 on dist-test-slave-42z9"
server_key: "e9fb69e2b4265127f919b9de9e173628"
server_key_iv: "59fe09d1a1c4647de0c2983cc199c0ea"
server_key_version: "encryptionkey@0"
I20260809 13:09:31.536674 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:31.551956 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:31.555477 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:31.555729 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:31.556581 15317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:31.557873 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:31.557952 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.558069 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:31.558121 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:31.586282 15317 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:43233
I20260809 13:09:31.586334 15525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:43233 every 8 connection(s)
I20260809 13:09:31.587627 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:31.589798 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 15317
I20260809 13:09:31.589998 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:31.590533 10966 external_mini_cluster.cc:1428] Setting key c3d143c89e0c7b0dd33393f4b43d1c02
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[14726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280971, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:31.606631 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:31.590114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:35147 (local address 127.10.181.190:38035)
0809 13:09:31.590278 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:31.590282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:31.591783 (+ 1501us) server_negotiation.cc:407] Connection header received
0809 13:09:31.593933 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:31.593937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:31.594006 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:31.594088 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:31.595446 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.596185 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.597644 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.598026 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.598127 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:31.601275 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:31.601280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:31.601284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:31.601316 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:31.603573 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:31.604283 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:31.604288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:31.604291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:31.604443 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:31.604939 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:31.604944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:31.604947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:31.605323 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:31.605486 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:31.606224 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 13:09:31.606383 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 13:09:31.607461 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:31.590770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:38035 (local address 127.10.181.131:35147)
0809 13:09:31.591609 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:31.591669 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:31.593628 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:31.594273 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:31.594280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:31.594783 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:31.595318 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:31.595336 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.596302 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.596306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:31.597529 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:31.597542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:31.598189 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:31.598194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:31.598390 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:31.598980 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:31.599003 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:31.601077 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:31.603706 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:31.603715 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:31.603729 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:31.604140 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:31.604561 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:31.604565 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:31.604569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:31.604825 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:31.605477 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:31.605485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:31.605881 (+ 396us) client_negotiation.cc:769] Sending connection context
0809 13:09:31.606131 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 13:09:31.606475 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":336,"threads_started":1}
I20260809 13:09:31.608641 15526 heartbeater.cc:347] Connected to a master server at 127.10.181.190:38035
I20260809 13:09:31.608909 15526 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:31.609681 15526 heartbeater.cc:511] Master 127.10.181.190:38035 requested a full tablet report, sending...
I20260809 13:09:31.611083 14796 ts_manager.cc:194] Registered new tserver with Master: 30eaae0500e6452fa15c627cd8bbd41a (127.10.181.131:43233)
I20260809 13:09:31.611852 14796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:35147
I20260809 13:09:31.617774 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:31.652077 15103 heartbeater.cc:503] Master 127.10.181.190:38035 was elected leader, sending a full tablet report...
W20260809 13:09:31.653070 15105 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:31.650610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.10.181.190:38035)
0809 13:09:31.650779 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:31.650784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:31.651201 (+ 417us) server_negotiation.cc:407] Connection header received
0809 13:09:31.651349 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:31.651354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:31.651415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:31.651504 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:31.652814 (+ 1310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39632: BlockingRecv error: recv got EOF from 127.0.0.1:39632 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:09:31.656693 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14900
W20260809 13:09:31.659246 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14900: Not authorized: Client connection negotiation failed: client connection to 127.10.181.129:39471: server requires authentication, but client does not have Kerberos credentials available
I20260809 13:09:31.659435 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14900
I20260809 13:09:31.683794 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15110
W20260809 13:09:31.686486 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15110: Not authorized: Client connection negotiation failed: client connection to 127.10.181.130:35853: server requires authentication, but client does not have Kerberos credentials available
I20260809 13:09:31.686635 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 15110
I20260809 13:09:31.712108 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15317
W20260809 13:09:31.714948 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15317: Not authorized: Client connection negotiation failed: client connection to 127.10.181.131:43233: server requires authentication, but client does not have Kerberos credentials available
I20260809 13:09:31.715102 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 15317
I20260809 13:09:31.735936 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14745
I20260809 13:09:31.737092 14849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:31.838868 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14745
2026-08-09T13:09:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3611 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[15553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[15553](info): set up 2 sockets
Aug 09 13:09:31 dist-test-slave-42z9 krb5kdc[15553](info): commencing operation
krb5kdc: starting...
W20260809 13:09:33.052486 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:33.242705 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:35839
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:35839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:33.554082 15576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:33.554324 15576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:33.554455 15576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:33.561187 15576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:33.561285 15576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:33.561360 15576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:33.561452 15576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:33.561523 15576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:33.570647 15576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:35839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:35839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:33.571704 15576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:33.572891 15576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:33.583714 15583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:33.583791 15584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:33.583982 15586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:33.584661 15576 server_base.cc:1034] running on GCE node
I20260809 13:09:33.585353 15576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:33.586619 15576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:33.587822 15576 hybrid_clock.cc:648] HybridClock initialized: now 1786280973587776 us; error 29 us; skew 500 ppm
I20260809 13:09:33.588129 15576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280973, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:33.594607 15576 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:33.595973 15576 webserver.cc:460] Webserver started at http://127.10.181.190:43645/ using document root <none> and password file <none>
I20260809 13:09:33.596596 15576 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:33.596710 15576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:33.596992 15576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:33.599776 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "69db7ad981384e97aa287fc842e7a462"
format_stamp: "Formatted at 2026-08-09 13:09:33 on dist-test-slave-42z9"
server_key: "efff7c4c372cd6d2e29a3ddab72187ad"
server_key_iv: "3227077cd0bf5def4ee4b208d48cf193"
server_key_version: "encryptionkey@0"
I20260809 13:09:33.600531 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "69db7ad981384e97aa287fc842e7a462"
format_stamp: "Formatted at 2026-08-09 13:09:33 on dist-test-slave-42z9"
server_key: "efff7c4c372cd6d2e29a3ddab72187ad"
server_key_iv: "3227077cd0bf5def4ee4b208d48cf193"
server_key_version: "encryptionkey@0"
I20260809 13:09:33.605221 15576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 13:09:33.608399 15597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:33.609688 15576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:33.609841 15576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "69db7ad981384e97aa287fc842e7a462"
format_stamp: "Formatted at 2026-08-09 13:09:33 on dist-test-slave-42z9"
server_key: "efff7c4c372cd6d2e29a3ddab72187ad"
server_key_iv: "3227077cd0bf5def4ee4b208d48cf193"
server_key_version: "encryptionkey@0"
I20260809 13:09:33.610002 15576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:33.621389 15576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:33.625181 15576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:33.625427 15576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:33.642071 15576 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:35839
I20260809 13:09:33.642109 15684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:35839 every 8 connection(s)
I20260809 13:09:33.643281 15576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:33.646715 15685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:33.647881 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 15576
I20260809 13:09:33.648136 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:33.648967 10966 external_mini_cluster.cc:1428] Setting key c5d556661d06fcf8c8b017f09d0bad87
I20260809 13:09:33.660689 15685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462: Bootstrap starting.
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:33.664124 15685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:33.665143 15685 log.cc:824] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:33.667697 15685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462: No bootstrap required, opened a new log
I20260809 13:09:33.670974 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:33.650803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.10.181.190:35839)
0809 13:09:33.651583 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:33.651594 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:33.651633 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:09:33.653187 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:33.653207 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:33.653608 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:33.653920 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:33.655732 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:33.656979 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:33.658726 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:33.659356 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:33.659490 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:33.663836 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:33.663843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:33.663857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:33.663894 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:33.666697 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:33.667323 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:33.667329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:33.667335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:33.667500 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:33.667861 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:33.667866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:33.667869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:33.668598 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:33.668909 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:33.669198 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 13:09:33.669650 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":386,"threads_started":1}
I20260809 13:09:33.674562 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:09:33.676932 15685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } }
I20260809 13:09:33.677367 15685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:33.677484 15685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69db7ad981384e97aa287fc842e7a462, State: Initialized, Role: FOLLOWER
I20260809 13:09:33.678103 15685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } }
I20260809 13:09:33.678371 15685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:33.678507 15685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:33.678684 15685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:33.681504 15685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } }
I20260809 13:09:33.682047 15685 leader_election.cc:302] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69db7ad981384e97aa287fc842e7a462; no voters:
I20260809 13:09:33.682698 15685 leader_election.cc:288] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:33.682776 15691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:33.683934 15691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [term 1 LEADER]: Becoming Leader. State: Replica: 69db7ad981384e97aa287fc842e7a462, State: Running, Role: LEADER
I20260809 13:09:33.684401 15691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } }
I20260809 13:09:33.684792 15685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:33.686944 15696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69db7ad981384e97aa287fc842e7a462. Latest consensus state: current_term: 1 leader_uuid: "69db7ad981384e97aa287fc842e7a462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } } }
I20260809 13:09:33.687427 15696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:33.687721 15693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69db7ad981384e97aa287fc842e7a462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69db7ad981384e97aa287fc842e7a462" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35839 } } }
I20260809 13:09:33.688025 15693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:33.702482 15704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:33.708364 15704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:33.719708 15704 catalog_manager.cc:1347] Generated new cluster ID: a3c09b3acd81415bb2c38759a9bd770b
I20260809 13:09:33.719816 15704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:33.738169 15704 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:33.740018 15704 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:33.757606 15704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69db7ad981384e97aa287fc842e7a462: Generated new TSK 0
I20260809 13:09:33.758553 15704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:09:33.761933 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:33.751563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.10.181.190:35839)
0809 13:09:33.751744 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:33.751751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:33.754030 (+ 2279us) server_negotiation.cc:407] Connection header received
0809 13:09:33.754251 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:33.754256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:33.754335 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:33.754431 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:33.755695 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:33.756610 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:33.758311 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:33.758965 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:33.759108 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:33.760292 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:33.760299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:33.760304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:33.761144 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:33.761217 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:33.761254 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:33.761366 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:33.761499 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 13:09:33.761604 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:09:33.768712 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15576
I20260809 13:09:33.770465 15679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:33.858260 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 15576
2026-08-09T13:09:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2014 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15743](info): set up 2 sockets
Aug 09 13:09:33 dist-test-slave-42z9 krb5kdc[15743](info): commencing operation
krb5kdc: starting...
W20260809 13:09:35.047031 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:35 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:35.240126 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:45249
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:45249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:35.558158 15766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:35.558429 15766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:35.558523 15766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:35.565423 15766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:35.565531 15766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:35.565582 15766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:35.565636 15766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:35.565680 15766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:35.574869 15766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:45249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:45249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:35.575999 15766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:35.577164 15766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:35.587915 15775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:35.587937 15772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:35.587910 15773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:35.588781 15766 server_base.cc:1034] running on GCE node
I20260809 13:09:35.589488 15766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:35.590752 15766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:35.591949 15766 hybrid_clock.cc:648] HybridClock initialized: now 1786280975591897 us; error 40 us; skew 500 ppm
I20260809 13:09:35.592259 15766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:35 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280975, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:35.595160 15766 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:35.596549 15766 webserver.cc:460] Webserver started at http://127.10.181.190:42339/ using document root <none> and password file <none>
I20260809 13:09:35.597149 15766 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:35.597256 15766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:35.597584 15766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:35.600499 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "c95269734fc248a5b9f8b5fda8ecd34c"
format_stamp: "Formatted at 2026-08-09 13:09:35 on dist-test-slave-42z9"
server_key: "7c677e2eb4f2a0d6bb9d428731d5a1e9"
server_key_iv: "908a72e8d84faa3cadcdb0973d5acce6"
server_key_version: "encryptionkey@0"
I20260809 13:09:35.601199 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "c95269734fc248a5b9f8b5fda8ecd34c"
format_stamp: "Formatted at 2026-08-09 13:09:35 on dist-test-slave-42z9"
server_key: "7c677e2eb4f2a0d6bb9d428731d5a1e9"
server_key_iv: "908a72e8d84faa3cadcdb0973d5acce6"
server_key_version: "encryptionkey@0"
I20260809 13:09:35.606027 15766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 13:09:35.609256 15787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:35.610519 15766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:35.610678 15766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "c95269734fc248a5b9f8b5fda8ecd34c"
format_stamp: "Formatted at 2026-08-09 13:09:35 on dist-test-slave-42z9"
server_key: "7c677e2eb4f2a0d6bb9d428731d5a1e9"
server_key_iv: "908a72e8d84faa3cadcdb0973d5acce6"
server_key_version: "encryptionkey@0"
I20260809 13:09:35.610842 15766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:35.624243 15766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:35.627750 15766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:35.627995 15766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:35.644799 15766 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:45249
I20260809 13:09:35.644831 15870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:45249 every 8 connection(s)
I20260809 13:09:35.646065 15766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:35.646603 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 15766
I20260809 13:09:35.646806 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:35.647545 10966 external_mini_cluster.cc:1428] Setting key 564d54049ed88afc91b768ad1bff8bc3
I20260809 13:09:35.650225 15873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:09:35 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:35.662691 15873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c: Bootstrap starting.
I20260809 13:09:35.665735 15873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:35.666922 15873 log.cc:824] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:35.669983 15873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c: No bootstrap required, opened a new log
I20260809 13:09:35.670353 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:35.649353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.10.181.190:45249)
0809 13:09:35.650149 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:35.650164 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:35.650234 (+ 70us) server_negotiation.cc:407] Connection header received
0809 13:09:35.652429 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:35.652456 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:35.652888 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:35.653300 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:35.654606 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:35.656601 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:35.657836 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:35.658798 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:35.658977 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:35.663143 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:35.663150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:35.663164 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:35.663203 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:35.666131 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:35.666932 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:35.666938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:35.666943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:35.667144 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:35.667610 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:35.667615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:35.667618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:35.668329 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:35.668551 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:35.668994 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 13:09:35.669336 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":317,"threads_started":1}
I20260809 13:09:35.678369 15873 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } }
I20260809 13:09:35.678741 15873 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:35.678841 15873 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95269734fc248a5b9f8b5fda8ecd34c, State: Initialized, Role: FOLLOWER
I20260809 13:09:35.679339 15873 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } }
I20260809 13:09:35.679561 15873 raft_consensus.cc:388] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:35.679687 15873 raft_consensus.cc:482] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:35.679844 15873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:35.681991 15873 raft_consensus.cc:504] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } }
I20260809 13:09:35.682440 15873 leader_election.cc:302] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c95269734fc248a5b9f8b5fda8ecd34c; no voters:
I20260809 13:09:35.682967 15873 leader_election.cc:288] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:35.683086 15881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:35.683722 15881 raft_consensus.cc:686] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [term 1 LEADER]: Becoming Leader. State: Replica: c95269734fc248a5b9f8b5fda8ecd34c, State: Running, Role: LEADER
I20260809 13:09:35.684240 15881 consensus_queue.cc:237] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } }
I20260809 13:09:35.684880 15873 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:35.687032 15885 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c95269734fc248a5b9f8b5fda8ecd34c. Latest consensus state: current_term: 1 leader_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } } }
I20260809 13:09:35.687321 15885 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:35.687780 15882 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95269734fc248a5b9f8b5fda8ecd34c" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45249 } } }
I20260809 13:09:35.688231 15882 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:35.689229 15894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:35.694502 15894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:35.703642 15894 catalog_manager.cc:1347] Generated new cluster ID: 9d9f64c165434b328cb68b3b8ca0dc06
I20260809 13:09:35.703743 15894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:35.727485 15894 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:35.728770 15894 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:35.738175 15894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c95269734fc248a5b9f8b5fda8ecd34c: Generated new TSK 0
I20260809 13:09:35.738940 15894 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:35.798817 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:36.125627 15920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:36.125911 15920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:36.126048 15920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:36.132965 15920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:36.133077 15920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:36.133239 15920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:36.142462 15920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:36.143669 15920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:36.144932 15920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:36.157475 15934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:36.157498 15935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:36.157414 15938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:36.158174 15920 server_base.cc:1034] running on GCE node
I20260809 13:09:36.158779 15920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:36.159675 15920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:36.160877 15920 hybrid_clock.cc:648] HybridClock initialized: now 1786280976160818 us; error 46 us; skew 500 ppm
I20260809 13:09:36.161180 15920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:36 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280976, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:36.164122 15920 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:36.165421 15920 webserver.cc:460] Webserver started at http://127.10.181.129:34401/ using document root <none> and password file <none>
I20260809 13:09:36.166098 15920 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:36.166217 15920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:36.166508 15920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:36.169278 15920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "225cf102987f4e38b7b964afb13a9585"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "0147c3d4d434cc5c20192243f55b2e7b"
server_key_iv: "a84c76beaed30c0e51728af8abde7886"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.170128 15920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "225cf102987f4e38b7b964afb13a9585"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "0147c3d4d434cc5c20192243f55b2e7b"
server_key_iv: "a84c76beaed30c0e51728af8abde7886"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.174978 15920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 13:09:36.178381 15945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.179677 15920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:09:36.179837 15920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "225cf102987f4e38b7b964afb13a9585"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "0147c3d4d434cc5c20192243f55b2e7b"
server_key_iv: "a84c76beaed30c0e51728af8abde7886"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.180022 15920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:36.197113 15920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:36.200740 15920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:36.200973 15920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:36.201889 15920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:36.203135 15920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:36.203205 15920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.203296 15920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:36.203339 15920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.232976 15920 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:36983
I20260809 13:09:36.233011 16133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:36983 every 8 connection(s)
I20260809 13:09:36.234186 15920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:36.238183 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 15920
I20260809 13:09:36.238389 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:36.239043 10966 external_mini_cluster.cc:1428] Setting key 2b6de9fefe1ee6760a330869df710451
Aug 09 13:09:36 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280976, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:36.253563 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:36.236549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:60459 (local address 127.10.181.190:45249)
0809 13:09:36.236988 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:36.236993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:36.238013 (+ 1020us) server_negotiation.cc:407] Connection header received
0809 13:09:36.239926 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:36.239931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:36.240024 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:36.240147 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:36.242154 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.242878 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.244384 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.244774 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.244878 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:36.248279 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:36.248283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:36.248287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:36.248318 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:36.250652 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:36.251327 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:36.251331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:36.251333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:36.251451 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:36.251931 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:36.251935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:36.251937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:36.252207 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:36.252324 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:36.253096 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 13:09:36.253263 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":226,"threads_started":1}
I20260809 13:09:36.254796 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:36.236977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:45249 (local address 127.10.181.129:60459)
0809 13:09:36.237796 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:36.237850 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:36.239645 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:36.240420 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:36.240431 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:36.241368 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:36.241986 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:36.242004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.243038 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.243042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:36.244239 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:36.244252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.245003 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.245012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:36.245212 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:36.245841 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:36.245868 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:36.248077 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:36.250791 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:36.250800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:36.250815 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:36.251203 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:36.251556 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:36.251560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:36.251563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:36.251788 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:36.252335 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:36.252343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:36.252723 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 13:09:36.252998 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 13:09:36.253501 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":318,"threads_started":1}
I20260809 13:09:36.256311 16134 heartbeater.cc:347] Connected to a master server at 127.10.181.190:45249
I20260809 13:09:36.256597 16134 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:36.257302 16134 heartbeater.cc:511] Master 127.10.181.190:45249 requested a full tablet report, sending...
I20260809 13:09:36.259349 15814 ts_manager.cc:194] Registered new tserver with Master: 225cf102987f4e38b7b964afb13a9585 (127.10.181.129:36983)
I20260809 13:09:36.260771 15814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:60459
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:36.286839 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:36.600354 16144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:36.600605 16144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:36.600705 16144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:36.607898 16144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:36.607997 16144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:36.608110 16144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:36.617383 16144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:36.618685 16144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:36.619940 16144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:36.632269 16154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:36.632292 16156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:36.632754 16152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:36.633002 16144 server_base.cc:1034] running on GCE node
I20260809 13:09:36.633719 16144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:36.634662 16144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:36.635864 16144 hybrid_clock.cc:648] HybridClock initialized: now 1786280976635809 us; error 42 us; skew 500 ppm
I20260809 13:09:36.636171 16144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:36 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280976, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:36.639274 16144 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:36.640627 16144 webserver.cc:460] Webserver started at http://127.10.181.130:37553/ using document root <none> and password file <none>
I20260809 13:09:36.641264 16144 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:36.641371 16144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:36.641688 16144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:36.644731 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "f673d168e4364d15acb770ee57ae5f38"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "1c4c64d36bd02c0a4ac427e5ea969921"
server_key_iv: "ca6fdfe234dd3ae0a78d54ff6a09b359"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.645761 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "f673d168e4364d15acb770ee57ae5f38"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "1c4c64d36bd02c0a4ac427e5ea969921"
server_key_iv: "ca6fdfe234dd3ae0a78d54ff6a09b359"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.652160 16144 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260809 13:09:36.656319 16168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.657804 16144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:36.657967 16144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "f673d168e4364d15acb770ee57ae5f38"
format_stamp: "Formatted at 2026-08-09 13:09:36 on dist-test-slave-42z9"
server_key: "1c4c64d36bd02c0a4ac427e5ea969921"
server_key_iv: "ca6fdfe234dd3ae0a78d54ff6a09b359"
server_key_version: "encryptionkey@0"
I20260809 13:09:36.658155 16144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:36.679687 16144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:36.683956 16144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:36.684244 16144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:36.685245 16144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:36.686627 16144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:36.686712 16144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.686836 16144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:36.686889 16144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:36.717173 16144 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43729
I20260809 13:09:36.717218 16362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43729 every 8 connection(s)
I20260809 13:09:36.718544 16144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:36.725122 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 16144
I20260809 13:09:36.725263 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:36.725724 10966 external_mini_cluster.cc:1428] Setting key 36664ef941fa062060ee0dcfc0bcb30b
Aug 09 13:09:36 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280976, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:36.737922 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:36.720996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51079 (local address 127.10.181.190:45249)
0809 13:09:36.721173 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:36.721178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:36.722369 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 13:09:36.724148 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:36.724154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:36.724238 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:36.724328 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:36.725953 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.726820 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.728416 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.728821 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.728932 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:36.732217 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:36.732222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:36.732226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:36.732257 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:36.734743 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:36.735446 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:36.735450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:36.735453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:36.735581 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:36.736110 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:36.736114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:36.736117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:36.736375 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:36.736485 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:36.737329 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 13:09:36.737574 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:09:36.738675 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:36.721467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:45249 (local address 127.10.181.130:51079)
0809 13:09:36.722208 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:36.722245 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:36.723880 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:36.724590 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:36.724598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:36.725098 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:36.725773 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:36.725797 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.726970 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.726974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:36.728275 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:36.728289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:36.729045 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:36.729052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:36.729275 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:36.729918 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:36.729942 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:36.732001 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:36.734875 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:36.734884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:36.734900 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:36.735323 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:36.735745 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:36.735750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:36.735754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:36.736003 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:36.736490 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:36.736498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:36.736915 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 13:09:36.737192 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 13:09:36.737617 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":271,"threads_started":1}
I20260809 13:09:36.740015 16364 heartbeater.cc:347] Connected to a master server at 127.10.181.190:45249
I20260809 13:09:36.740300 16364 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:36.740993 16364 heartbeater.cc:511] Master 127.10.181.190:45249 requested a full tablet report, sending...
I20260809 13:09:36.742442 15814 ts_manager.cc:194] Registered new tserver with Master: f673d168e4364d15acb770ee57ae5f38 (127.10.181.130:43729)
I20260809 13:09:36.743228 15814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:51079
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:36.772339 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:37.100689 16371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:37.100948 16371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:37.101043 16371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:37.107947 16371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:37.108040 16371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:37.108162 16371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:37.117465 16371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:45249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:37.118615 16371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:37.120015 16371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:37.132706 16376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:37.132694 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:37.132740 16378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:37.133684 16371 server_base.cc:1034] running on GCE node
I20260809 13:09:37.134240 16371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:37.135090 16371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:37.136283 16371 hybrid_clock.cc:648] HybridClock initialized: now 1786280977136233 us; error 46 us; skew 500 ppm
I20260809 13:09:37.136584 16371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:37.139668 16371 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:37.140967 16371 webserver.cc:460] Webserver started at http://127.10.181.131:39139/ using document root <none> and password file <none>
I20260809 13:09:37.141664 16371 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:37.141782 16371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:37.142057 16371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:37.145133 16371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "647bbbd5f4ed4e569eaf2c0318f49074"
format_stamp: "Formatted at 2026-08-09 13:09:37 on dist-test-slave-42z9"
server_key: "cad2278757b1e46e5d04669a227fa079"
server_key_iv: "4f5852b62a7e230cf43230cd1e561226"
server_key_version: "encryptionkey@0"
I20260809 13:09:37.145938 16371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "647bbbd5f4ed4e569eaf2c0318f49074"
format_stamp: "Formatted at 2026-08-09 13:09:37 on dist-test-slave-42z9"
server_key: "cad2278757b1e46e5d04669a227fa079"
server_key_iv: "4f5852b62a7e230cf43230cd1e561226"
server_key_version: "encryptionkey@0"
I20260809 13:09:37.151079 16371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 13:09:37.158805 16393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:37.162168 16371 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260809 13:09:37.162336 16371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "647bbbd5f4ed4e569eaf2c0318f49074"
format_stamp: "Formatted at 2026-08-09 13:09:37 on dist-test-slave-42z9"
server_key: "cad2278757b1e46e5d04669a227fa079"
server_key_iv: "4f5852b62a7e230cf43230cd1e561226"
server_key_version: "encryptionkey@0"
I20260809 13:09:37.162561 16371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:37.181118 16371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:37.186431 16371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:37.187784 16371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:37.188935 16371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:37.190632 16371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:37.190723 16371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:37.190851 16371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:37.190920 16371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:37.221239 16371 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:43077
I20260809 13:09:37.221277 16580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:43077 every 8 connection(s)
I20260809 13:09:37.222536 16371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:37.231330 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 16371
I20260809 13:09:37.231491 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:37.231907 10966 external_mini_cluster.cc:1428] Setting key e0f80dad7d9bce44772e4cb008558a53
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:37.242738 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.224979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:54547 (local address 127.10.181.190:45249)
0809 13:09:37.225207 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:37.225214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:37.226400 (+ 1186us) server_negotiation.cc:407] Connection header received
0809 13:09:37.228172 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:37.228176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:37.228247 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:37.228332 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:37.229803 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.230538 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.232169 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.232754 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.232868 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.237608 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:37.237613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:37.237616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:37.237650 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:37.240181 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.240824 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.240827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.240830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.240940 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.241300 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.241303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.241305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.241615 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:37.241724 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:37.242334 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 13:09:37.242491 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:09:37.243643 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.225503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:45249 (local address 127.10.181.131:54547)
0809 13:09:37.226232 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:37.226278 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:37.227919 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:37.228547 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:37.228555 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:37.229047 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:37.229663 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:37.229683 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.230704 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.230708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:37.232001 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:37.232017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.232994 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.233001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:37.233303 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.234679 (+ 1376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:37.234709 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:37.237369 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:37.240318 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:37.240325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:37.240335 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:37.240732 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:37.241011 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:37.241014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:37.241016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:37.241205 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:37.241723 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:37.241728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:37.242020 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 13:09:37.242253 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 13:09:37.242617 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":270,"threads_started":1}
I20260809 13:09:37.244854 16582 heartbeater.cc:347] Connected to a master server at 127.10.181.190:45249
I20260809 13:09:37.245163 16582 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:37.246060 16582 heartbeater.cc:511] Master 127.10.181.190:45249 requested a full tablet report, sending...
I20260809 13:09:37.247470 15814 ts_manager.cc:194] Registered new tserver with Master: 647bbbd5f4ed4e569eaf2c0318f49074 (127.10.181.131:43077)
I20260809 13:09:37.248246 15814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:54547
I20260809 13:09:37.258947 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:37.263487 16134 heartbeater.cc:503] Master 127.10.181.190:45249 was elected leader, sending a full tablet report...
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:37.311859 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.298534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.10.181.190:45249)
0809 13:09:37.298750 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:37.298756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:37.299154 (+ 398us) server_negotiation.cc:407] Connection header received
0809 13:09:37.299343 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:37.299348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:37.299429 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:37.299515 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:37.300858 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.301655 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.302897 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.303306 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.303419 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.307038 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:37.307042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:37.307046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:37.307081 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:37.309310 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.309928 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.309932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.309935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.310065 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.310426 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.310430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.310432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.310761 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:37.311011 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:37.311335 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 13:09:37.311552 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 13:09:37.312466 15814 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48036
I20260809 13:09:37.314719 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15920
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:37.329092 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.315823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.10.181.129:36983)
0809 13:09:37.316422 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:37.316429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:37.316453 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:09:37.316509 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:37.316515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:37.316710 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:37.316948 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:37.317938 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.318920 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.320119 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.320671 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.320769 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.324208 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:37.324220 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:37.324234 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:37.324266 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:37.326817 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.327482 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.327487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.327490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.327612 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.327897 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.327900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.327902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.328207 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:37.328429 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:37.328604 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 13:09:37.328797 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":296,"threads_started":1}
W20260809 13:09:37.330340 16126 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43198
W20260809 13:09:37.331321 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15920: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:37.331513 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 15920
I20260809 13:09:37.354628 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16144
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:37.369863 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.355533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.10.181.130:43729)
0809 13:09:37.356023 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:37.356031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:37.356056 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:09:37.356134 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:37.356140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:37.356352 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:37.356477 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:37.357718 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.358718 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.359878 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.360403 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.360494 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.364512 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:37.364523 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:37.364538 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:37.364582 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:37.367183 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.367759 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.367765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.367768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.367887 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.368230 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.368234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.368235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.368586 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:37.368818 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:37.369087 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 13:09:37.369340 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":234,"threads_started":1}
W20260809 13:09:37.371102 16355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43158
W20260809 13:09:37.372164 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16144: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:37.372308 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 16144
I20260809 13:09:37.397980 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16371
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[15743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280977, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:37.414160 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:37.398897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.10.181.131:43077)
0809 13:09:37.399535 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:37.399542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:37.399572 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:09:37.399644 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:37.399651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:37.399863 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:37.400014 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:37.401409 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.402547 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.403812 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:37.404369 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:37.404486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:37.408631 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:37.408643 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:37.408658 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:37.408693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:37.411367 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.412005 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.412013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.412015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.412148 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:37.412479 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:37.412483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:37.412485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:37.412813 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:37.413118 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:37.413409 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 13:09:37.413845 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":297,"threads_started":1}
W20260809 13:09:37.415386 16576 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52088
W20260809 13:09:37.416383 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16371: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:09:37.416541 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 16371
I20260809 13:09:37.452667 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 15766
I20260809 13:09:37.453816 15864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:37.556488 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 15766
2026-08-09T13:09:37Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3699 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[16611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[16611](info): set up 2 sockets
Aug 09 13:09:37 dist-test-slave-42z9 krb5kdc[16611](info): commencing operation
krb5kdc: starting...
W20260809 13:09:38.746402 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:38 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:38.994164 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39883
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:39.320479 16628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:39.320763 16628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:39.320865 16628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:39.327943 16628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:39.328050 16628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:39.328104 16628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:39.328158 16628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:39.328202 16628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:39.337598 16628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:39.338792 16628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:39.340078 16628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:39.351063 16641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:39.351042 16637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:39.351542 16636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:39.351974 16628 server_base.cc:1034] running on GCE node
I20260809 13:09:39.352674 16628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:39.354007 16628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:39.355211 16628 hybrid_clock.cc:648] HybridClock initialized: now 1786280979355166 us; error 42 us; skew 500 ppm
I20260809 13:09:39.355531 16628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:39 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280979, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:39.362195 16628 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:39.363610 16628 webserver.cc:460] Webserver started at http://127.10.181.190:43843/ using document root <none> and password file <none>
I20260809 13:09:39.364262 16628 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:39.364378 16628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:39.364691 16628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:39.367601 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "c7b0291bca7045e8a15ac9a42aaac293"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "78e947d546053f817f5b0f2dc3c32eba"
server_key_iv: "fc30ca1d239f55fb7a8baab6be8f6e26"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.368353 16628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "c7b0291bca7045e8a15ac9a42aaac293"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "78e947d546053f817f5b0f2dc3c32eba"
server_key_iv: "fc30ca1d239f55fb7a8baab6be8f6e26"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.373219 16628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 13:09:39.376520 16650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:39.377918 16628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:39.378082 16628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "c7b0291bca7045e8a15ac9a42aaac293"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "78e947d546053f817f5b0f2dc3c32eba"
server_key_iv: "fc30ca1d239f55fb7a8baab6be8f6e26"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.378257 16628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:39.392273 16628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:39.395941 16628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:39.396206 16628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:39.413656 16628 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39883
I20260809 13:09:39.413645 16745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39883 every 8 connection(s)
I20260809 13:09:39.415016 16628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:39.418695 16746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:39.424001 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 16628
I20260809 13:09:39.424352 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:39.424988 10966 external_mini_cluster.cc:1428] Setting key 52c36dff6c2f15ab55712507e9e90490
I20260809 13:09:39.431847 16746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293: Bootstrap starting.
I20260809 13:09:39.435627 16746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:39.436750 16746 log.cc:824] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:09:39 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:39.453056 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:39.427085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.10.181.190:39883)
0809 13:09:39.434423 (+ 7338us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:39.434438 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:39.434489 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:09:39.436411 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:39.436437 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:39.436835 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:39.437176 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:39.439042 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:39.440327 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:39.441753 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:39.442381 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:39.442512 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:39.446798 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:39.446806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:39.446822 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:39.446867 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:39.449430 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:39.450032 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:39.450038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:39.450044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:39.450199 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:39.450483 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:39.450486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:39.450488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:39.451011 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:39.451230 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:39.451722 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 13:09:39.452046 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7101,"thread_start_us":6862,"threads_started":1}
I20260809 13:09:39.462148 16746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293: No bootstrap required, opened a new log
I20260809 13:09:39.469987 16746 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } }
I20260809 13:09:39.470362 16746 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:39.470438 16746 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b0291bca7045e8a15ac9a42aaac293, State: Initialized, Role: FOLLOWER
I20260809 13:09:39.470975 16746 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } }
I20260809 13:09:39.471189 16746 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:39.471297 16746 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:39.471423 16746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:39.473845 16746 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } }
I20260809 13:09:39.474320 16746 leader_election.cc:302] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7b0291bca7045e8a15ac9a42aaac293; no voters:
I20260809 13:09:39.474870 16746 leader_election.cc:288] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:39.474995 16751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:39.475682 16751 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [term 1 LEADER]: Becoming Leader. State: Replica: c7b0291bca7045e8a15ac9a42aaac293, State: Running, Role: LEADER
I20260809 13:09:39.476210 16751 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } }
I20260809 13:09:39.476912 16746 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:39.479082 16754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7b0291bca7045e8a15ac9a42aaac293. Latest consensus state: current_term: 1 leader_uuid: "c7b0291bca7045e8a15ac9a42aaac293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } } }
I20260809 13:09:39.479055 16752 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7b0291bca7045e8a15ac9a42aaac293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b0291bca7045e8a15ac9a42aaac293" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39883 } } }
I20260809 13:09:39.479578 16754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:39.479578 16752 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:39.481839 16761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:39.487600 16761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:39.496460 16761 catalog_manager.cc:1347] Generated new cluster ID: 4d201cb39af24c18a2cf6e9149ec1345
I20260809 13:09:39.496577 16761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:39.512975 16761 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:39.514822 16761 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:39.528612 16761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7b0291bca7045e8a15ac9a42aaac293: Generated new TSK 0
I20260809 13:09:39.529685 16761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:39.586939 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:39.926890 16796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:39.927140 16796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:39.927273 16796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:39.934221 16796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:39.934314 16796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:39.934461 16796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:39.943821 16796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:39.945008 16796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:39.946317 16796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:39.958582 16809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:39.958546 16807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:39.958632 16806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:39.959728 16796 server_base.cc:1034] running on GCE node
I20260809 13:09:39.960296 16796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:39.961163 16796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:39.962348 16796 hybrid_clock.cc:648] HybridClock initialized: now 1786280979962297 us; error 39 us; skew 500 ppm
I20260809 13:09:39.962648 16796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:39 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280979, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:39.965624 16796 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:39.967011 16796 webserver.cc:460] Webserver started at http://127.10.181.129:40571/ using document root <none> and password file <none>
I20260809 13:09:39.967645 16796 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:39.967772 16796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:39.968078 16796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:39.970985 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "568f80fb9d3c4820b7c6370d4d94a879"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "f87cd8e679f908c8da5faa5768b4eaa7"
server_key_iv: "ad3c6ea10df15509f6bb9ce76317c7d9"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.971747 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "568f80fb9d3c4820b7c6370d4d94a879"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "f87cd8e679f908c8da5faa5768b4eaa7"
server_key_iv: "ad3c6ea10df15509f6bb9ce76317c7d9"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.976873 16796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:09:39.980396 16821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:39.981746 16796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:39.981896 16796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "568f80fb9d3c4820b7c6370d4d94a879"
format_stamp: "Formatted at 2026-08-09 13:09:39 on dist-test-slave-42z9"
server_key: "f87cd8e679f908c8da5faa5768b4eaa7"
server_key_iv: "ad3c6ea10df15509f6bb9ce76317c7d9"
server_key_version: "encryptionkey@0"
I20260809 13:09:39.982087 16796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:39.993903 16796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:39.997565 16796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:39.997822 16796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:39.998768 16796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:40.000072 16796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:40.000159 16796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.000277 16796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:40.000331 16796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.030718 16796 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:41327
I20260809 13:09:40.030763 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:41327 every 8 connection(s)
I20260809 13:09:40.031941 16796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:40.039515 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 16796
I20260809 13:09:40.039740 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:40.040354 10966 external_mini_cluster.cc:1428] Setting key d256f2cc53d322e2f075807d429ec08d
Aug 09 13:09:40 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280979, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:40.050567 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:40.034349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:48675 (local address 127.10.181.190:39883)
0809 13:09:40.034767 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:40.034772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:40.035690 (+ 918us) server_negotiation.cc:407] Connection header received
0809 13:09:40.037488 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:40.037494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:40.037571 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:40.037670 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:40.039148 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.039944 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.041503 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.041916 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.042030 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:40.045310 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:40.045315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:40.045319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:40.045350 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:40.047660 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:40.048331 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:40.048336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:40.048338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:40.048459 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:40.048870 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:40.048875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:40.048878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:40.049140 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:40.049267 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:40.049979 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 13:09:40.050209 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":202,"threads_started":1}
I20260809 13:09:40.051460 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:40.034855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39883 (local address 127.10.181.129:48675)
0809 13:09:40.035526 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:40.035579 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:40.037209 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:40.037905 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:40.037913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:40.038401 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:40.038970 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:40.038988 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.040092 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.040096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:40.041334 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:40.041346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.042137 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.042146 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:40.042365 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:40.043023 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:40.043045 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:40.045117 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:40.047790 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:40.047799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:40.047815 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:40.048219 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:40.048579 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:40.048583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:40.048585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:40.048763 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:40.049274 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:40.049281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:40.049632 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 13:09:40.049868 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 13:09:40.050363 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":258,"threads_started":1}
I20260809 13:09:40.052973 17008 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39883
I20260809 13:09:40.053265 17008 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:40.053992 17008 heartbeater.cc:511] Master 127.10.181.190:39883 requested a full tablet report, sending...
I20260809 13:09:40.056001 16693 ts_manager.cc:194] Registered new tserver with Master: 568f80fb9d3c4820b7c6370d4d94a879 (127.10.181.129:41327)
I20260809 13:09:40.057484 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:48675
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:40.088335 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:40.423722 17016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:40.423990 17016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:40.424132 17016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:40.431218 17016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:40.431334 17016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:40.431478 17016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:40.440770 17016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:40.442035 17016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:40.443300 17016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:40.455973 17027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:40.456048 17038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:40.456000 17028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:40.456852 17016 server_base.cc:1034] running on GCE node
I20260809 13:09:40.457472 17016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:40.458380 17016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:40.459579 17016 hybrid_clock.cc:648] HybridClock initialized: now 1786280980459529 us; error 42 us; skew 500 ppm
I20260809 13:09:40.459877 17016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:40 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280980, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:40.462857 17016 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:40.464215 17016 webserver.cc:460] Webserver started at http://127.10.181.130:40727/ using document root <none> and password file <none>
I20260809 13:09:40.464872 17016 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:40.464987 17016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:40.465301 17016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:40.468295 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "6fed595388d04486a457e7982726a021"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "153e351a288298c8b643f72a47648dab"
server_key_iv: "3b87ed15d925c9e63f32fc80daf8d19d"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.469043 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "6fed595388d04486a457e7982726a021"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "153e351a288298c8b643f72a47648dab"
server_key_iv: "3b87ed15d925c9e63f32fc80daf8d19d"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.473982 17016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 13:09:40.477394 17048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.478804 17016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:40.478961 17016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "6fed595388d04486a457e7982726a021"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "153e351a288298c8b643f72a47648dab"
server_key_iv: "3b87ed15d925c9e63f32fc80daf8d19d"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.479147 17016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:40.493959 17016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:40.497640 17016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:40.497896 17016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:40.498824 17016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:40.500129 17016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:40.500216 17016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.500332 17016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:40.500389 17016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.530499 17016 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:35045
I20260809 13:09:40.530535 17219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:35045 every 8 connection(s)
I20260809 13:09:40.531776 17016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:40.536737 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 17016
I20260809 13:09:40.536916 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:40.537324 10966 external_mini_cluster.cc:1428] Setting key 3f141f3002a8b2e29c69dd006d4ea781
Aug 09 13:09:40 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280980, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:40.551404 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:40.534280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:58989 (local address 127.10.181.190:39883)
0809 13:09:40.534454 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:40.534459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:40.535549 (+ 1090us) server_negotiation.cc:407] Connection header received
0809 13:09:40.537414 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:40.537420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:40.537534 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:40.537627 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:40.539220 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.540256 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.541781 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.542352 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.542473 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:40.545656 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:40.545662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:40.545667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:40.545704 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:40.548489 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:40.549280 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:40.549284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:40.549286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:40.549406 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:40.549868 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:40.549871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:40.549873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:40.550134 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:40.550242 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:40.550992 (+ 750us) server_negotiation.cc:299] Negotiation successful
0809 13:09:40.551166 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:09:40.552678 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:40.534692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39883 (local address 127.10.181.130:58989)
0809 13:09:40.535391 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:40.535432 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:40.537156 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:40.537896 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:40.537907 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:40.538462 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:40.539042 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:40.539068 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.540398 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.540401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:40.541644 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:40.541656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:40.542529 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:40.542533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:40.542739 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:40.543347 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:40.543369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:40.545404 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:40.548621 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:40.548629 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:40.548644 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:40.549185 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:40.549525 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:40.549529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:40.549532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:40.549776 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:40.550255 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:40.550262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:40.550644 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 13:09:40.550913 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 13:09:40.551369 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":266,"threads_started":1}
I20260809 13:09:40.554104 17220 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39883
I20260809 13:09:40.554390 17220 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:40.555073 17220 heartbeater.cc:511] Master 127.10.181.190:39883 requested a full tablet report, sending...
I20260809 13:09:40.556459 16693 ts_manager.cc:194] Registered new tserver with Master: 6fed595388d04486a457e7982726a021 (127.10.181.130:35045)
I20260809 13:09:40.557238 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:58989
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:40.586403 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:40.901377 17230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:40.901648 17230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:40.901777 17230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:40.908663 17230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:40.908768 17230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:40.908921 17230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:40.918159 17230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:40.919346 17230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:40.920595 17230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:40.932963 17238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:40.932929 17239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:40.933388 17246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:40.933856 17230 server_base.cc:1034] running on GCE node
I20260809 13:09:40.934418 17230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:40.935245 17230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:40.936441 17230 hybrid_clock.cc:648] HybridClock initialized: now 1786280980936404 us; error 30 us; skew 500 ppm
I20260809 13:09:40.936728 17230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:40 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280980, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:40.939682 17230 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:40.941004 17230 webserver.cc:460] Webserver started at http://127.10.181.131:35441/ using document root <none> and password file <none>
I20260809 13:09:40.941684 17230 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:40.941800 17230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:40.942094 17230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:40.944805 17230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "d34da4151b87421ea13cb115332d9e75"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "6154c8b167d5984724fd80475b12f49f"
server_key_iv: "d7849944a8f01acf198a620551f6841b"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.945585 17230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "d34da4151b87421ea13cb115332d9e75"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "6154c8b167d5984724fd80475b12f49f"
server_key_iv: "d7849944a8f01acf198a620551f6841b"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.950322 17230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 13:09:40.953567 17259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.954739 17230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 13:09:40.954888 17230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "d34da4151b87421ea13cb115332d9e75"
format_stamp: "Formatted at 2026-08-09 13:09:40 on dist-test-slave-42z9"
server_key: "6154c8b167d5984724fd80475b12f49f"
server_key_iv: "d7849944a8f01acf198a620551f6841b"
server_key_version: "encryptionkey@0"
I20260809 13:09:40.955061 17230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:40.983392 17230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:40.987562 17230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:40.987828 17230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:40.988775 17230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:40.990144 17230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:40.990221 17230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:40.990339 17230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:40.990404 17230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:41.020527 17230 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:45135
I20260809 13:09:41.020581 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:45135 every 8 connection(s)
I20260809 13:09:41.021757 17230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:41.022704 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 17230
I20260809 13:09:41.022899 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:41.023399 10966 external_mini_cluster.cc:1428] Setting key 4b7ee29b4dffb26d0ed7aa6d7138deb5
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280980, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:41.042596 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.024367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:42561 (local address 127.10.181.190:39883)
0809 13:09:41.024538 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.024542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.025980 (+ 1438us) server_negotiation.cc:407] Connection header received
0809 13:09:41.027998 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.028002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.028077 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.028151 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.029712 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.030469 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.032674 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.033043 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.033134 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.036889 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.036894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.036898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.036930 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.040021 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.040668 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.040671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.040674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.040798 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.041176 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.041179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.041180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.041433 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.041583 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.042182 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.042343 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:09:41.043414 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.024962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39883 (local address 127.10.181.131:42561)
0809 13:09:41.025831 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:41.025875 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:41.027749 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:41.028332 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:41.028340 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:41.028874 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:41.029575 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:41.029600 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.030605 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.030611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:41.032553 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:41.032568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.033288 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.033297 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:41.033604 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.034281 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:41.034309 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:41.036709 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:41.040159 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:41.040165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:41.040175 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:41.040579 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:41.040890 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:41.040893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:41.040896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:41.041089 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:41.041560 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:41.041565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:41.041854 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 13:09:41.042085 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 13:09:41.042431 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":363,"threads_started":1}
I20260809 13:09:41.044646 17468 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39883
I20260809 13:09:41.044909 17468 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:41.045610 17468 heartbeater.cc:511] Master 127.10.181.190:39883 requested a full tablet report, sending...
I20260809 13:09:41.046897 16693 ts_manager.cc:194] Registered new tserver with Master: d34da4151b87421ea13cb115332d9e75 (127.10.181.131:45135)
I20260809 13:09:41.047673 16693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:42561
I20260809 13:09:41.050938 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:41.060184 17008 heartbeater.cc:503] Master 127.10.181.190:39883 was elected leader, sending a full tablet report...
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:41.105120 17478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.091309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.10.181.129:41327)
0809 13:09:41.091962 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.091969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.091997 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:09:41.092060 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.092067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.092281 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.092413 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.093769 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.094795 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.095916 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.096452 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.096545 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.100154 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.100167 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.100181 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.100219 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.102856 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.103440 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.103446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.103448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.103572 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.103827 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.103830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.103832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.104157 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.104384 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.104555 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.104771 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":345,"threads_started":1}
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:41.121104 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.109161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49918 (local address 127.10.181.190:39883)
0809 13:09:41.109380 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.109386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.109523 (+ 137us) server_negotiation.cc:407] Connection header received
0809 13:09:41.109649 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.109653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.109716 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.109790 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.110883 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.111590 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.112734 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.113155 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.113243 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.116613 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.116618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.116622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.116653 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.118984 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.119524 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.119527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.119529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.119673 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.119963 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.119966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.119968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.120277 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.120456 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.120660 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.120814 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 13:09:41.121676 16693 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49918
I20260809 13:09:41.122570 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16796
I20260809 13:09:41.133546 17478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.123328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.10.181.129:41327)
0809 13:09:41.123530 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.123536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.123740 (+ 204us) server_negotiation.cc:407] Connection header received
0809 13:09:41.123844 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.123847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.123907 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.123978 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.124960 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.125638 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.126732 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.127186 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.127274 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.129316 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.129321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.129324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.129355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.131585 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.132117 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.132120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.132122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.132230 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.132492 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.132495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.132497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.132729 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.132898 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.133084 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.133235 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:09:41.134081 16997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:41.218003 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 16796
I20260809 13:09:41.240736 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 17016
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:41.255316 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.241634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.10.181.130:35045)
0809 13:09:41.242202 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.242210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.242239 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:09:41.242303 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.242310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.242534 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.242675 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.243737 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.244757 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.246030 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.246585 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.246684 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.250405 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.250417 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.250432 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.250466 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.253034 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.253698 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.253705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.253708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.253826 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.254098 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.254101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.254102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.254410 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.254646 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.254831 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.255026 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260809 13:09:41.256531 17214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:41.335991 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 17016
I20260809 13:09:41.359444 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 17230
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:41.374349 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:41.360356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.10.181.131:45135)
0809 13:09:41.360981 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:41.360988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:41.361017 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:09:41.361079 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:41.361086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:41.361297 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:41.361422 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:41.362586 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.363613 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.364811 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:41.365353 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:41.365567 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:41.369398 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:41.369410 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:41.369424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:41.369486 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:41.372123 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.372654 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.372660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.372663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.372774 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:41.373021 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:41.373024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:41.373026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:41.373336 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:41.373620 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:41.373806 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 13:09:41.374001 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":301,"threads_started":1}
I20260809 13:09:41.375510 17461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:41.462970 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 17230
I20260809 13:09:41.483312 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 16628
I20260809 13:09:41.484534 16741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:41.586918 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 16628
2026-08-09T13:09:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4032 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[17497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[17497](info): set up 2 sockets
Aug 09 13:09:41 dist-test-slave-42z9 krb5kdc[17497](info): commencing operation
krb5kdc: starting...
W20260809 13:09:42.781841 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:42 dist-test-slave-42z9 krb5kdc[17497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:42.982517 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33161
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:43.306704 17517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:43.306957 17517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:43.307093 17517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:43.314055 17517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:43.314152 17517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:43.314224 17517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:43.314297 17517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:43.314361 17517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:43.323781 17517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:43.324934 17517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:43.326191 17517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:43.337380 17535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:43.337299 17533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:43.337816 17532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:43.337783 17517 server_base.cc:1034] running on GCE node
I20260809 13:09:43.338737 17517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:43.340092 17517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:43.341300 17517 hybrid_clock.cc:648] HybridClock initialized: now 1786280983341247 us; error 46 us; skew 500 ppm
I20260809 13:09:43.341660 17517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:43 dist-test-slave-42z9 krb5kdc[17497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280983, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:43.344573 17517 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:43.345772 17517 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:43.345862 17517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:43.346144 17517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:43.348917 17517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "1908818a653f4c34940fe18321f8dcfa"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "21b3a7b682bb327fb6b3c8367638f015"
server_key_iv: "de6b2450467e900f55b89839cd139305"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.349722 17517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "1908818a653f4c34940fe18321f8dcfa"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "21b3a7b682bb327fb6b3c8367638f015"
server_key_iv: "de6b2450467e900f55b89839cd139305"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.354507 17517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:09:43.357780 17544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:43.358958 17517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:09:43.359129 17517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "1908818a653f4c34940fe18321f8dcfa"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "21b3a7b682bb327fb6b3c8367638f015"
server_key_iv: "de6b2450467e900f55b89839cd139305"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.359321 17517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:43.392863 17517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:43.396548 17517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:43.396786 17517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:43.413395 17517 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:33161
I20260809 13:09:43.413429 17627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:33161 every 8 connection(s)
I20260809 13:09:43.414264 17517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:43.417793 17634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:43.421552 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 17517
I20260809 13:09:43.421785 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:43.422565 10966 external_mini_cluster.cc:1428] Setting key 0b998d9ca89118559c99e21c5c12da3f
I20260809 13:09:43.430864 17634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: Bootstrap starting.
I20260809 13:09:43.434592 17634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:43.435870 17634 log.cc:824] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: Log is configured to *not* fsync() on all Append() calls
Aug 09 13:09:43 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:43.438897 17634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: No bootstrap required, opened a new log
I20260809 13:09:43.444947 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:43.424572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41306 (local address 127.10.181.190:33161)
0809 13:09:43.425331 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:43.425342 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:43.425384 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:09:43.426942 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:43.426965 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:43.427328 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:43.427650 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:43.429412 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:43.430685 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:43.432427 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:43.433081 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:43.433219 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:43.438039 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:43.438047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:43.438061 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:43.438095 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:43.441062 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:43.441851 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:43.441857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:43.441863 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:43.442029 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:43.442442 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:43.442445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:43.442447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:43.442962 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:43.443188 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:43.443610 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 13:09:43.443926 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":360,"threads_started":1}
I20260809 13:09:43.447522 17634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } }
I20260809 13:09:43.447898 17634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:43.447994 17634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1908818a653f4c34940fe18321f8dcfa, State: Initialized, Role: FOLLOWER
I20260809 13:09:43.448491 17634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } }
I20260809 13:09:43.448711 17634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:43.448841 17634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:43.448993 17634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:43.451071 17634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } }
I20260809 13:09:43.451519 17634 leader_election.cc:302] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1908818a653f4c34940fe18321f8dcfa; no voters:
I20260809 13:09:43.452028 17634 leader_election.cc:288] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:43.452140 17641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:43.453316 17641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [term 1 LEADER]: Becoming Leader. State: Replica: 1908818a653f4c34940fe18321f8dcfa, State: Running, Role: LEADER
I20260809 13:09:43.453850 17641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } }
I20260809 13:09:43.454252 17634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:43.456401 17644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1908818a653f4c34940fe18321f8dcfa. Latest consensus state: current_term: 1 leader_uuid: "1908818a653f4c34940fe18321f8dcfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } } }
I20260809 13:09:43.456811 17644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:43.456841 17643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1908818a653f4c34940fe18321f8dcfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1908818a653f4c34940fe18321f8dcfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33161 } } }
I20260809 13:09:43.457269 17643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:43.459051 17655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:43.463269 17655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:43.471343 17655 catalog_manager.cc:1347] Generated new cluster ID: 53ec6a8afc7242fdba07ac59b24718be
I20260809 13:09:43.471508 17655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:43.493170 17655 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:43.494400 17655 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:43.502974 17655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: Generated new TSK 0
I20260809 13:09:43.503702 17655 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:43.558754 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:43.883414 17676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:43.883663 17676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:43.883798 17676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:43.890990 17676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:43.891094 17676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:43.891255 17676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:43.900621 17676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:43.901839 17676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:43.903103 17676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:43.915410 17685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:43.915385 17684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:43.915705 17690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:43.916086 17676 server_base.cc:1034] running on GCE node
I20260809 13:09:43.916688 17676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:43.917613 17676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:43.918824 17676 hybrid_clock.cc:648] HybridClock initialized: now 1786280983918771 us; error 42 us; skew 500 ppm
I20260809 13:09:43.919135 17676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:43 dist-test-slave-42z9 krb5kdc[17497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280983, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:43.922211 17676 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:43.923234 17676 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:43.923321 17676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:43.923597 17676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:43.926540 17676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "b9cb45fde49e433990acdd0c897eaf9b"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "8658c448aa64446adacc54d25c4db5a3"
server_key_iv: "233899e56b52b402c49c80e17275729a"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.927284 17676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "b9cb45fde49e433990acdd0c897eaf9b"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "8658c448aa64446adacc54d25c4db5a3"
server_key_iv: "233899e56b52b402c49c80e17275729a"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.932128 17676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:43.935518 17696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:43.936785 17676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:09:43.936928 17676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "b9cb45fde49e433990acdd0c897eaf9b"
format_stamp: "Formatted at 2026-08-09 13:09:43 on dist-test-slave-42z9"
server_key: "8658c448aa64446adacc54d25c4db5a3"
server_key_iv: "233899e56b52b402c49c80e17275729a"
server_key_version: "encryptionkey@0"
I20260809 13:09:43.937134 17676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:43.966809 17676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:43.970854 17676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:43.971115 17676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:43.971992 17676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:43.973275 17676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:43.973363 17676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:43.973482 17676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:43.973567 17676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.003784 17676 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:44495
I20260809 13:09:44.003832 17901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:44495 every 8 connection(s)
I20260809 13:09:44.004608 17676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:44.007484 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 17676
I20260809 13:09:44.007686 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:44.008509 10966 external_mini_cluster.cc:1428] Setting key ac72ee62804e6e40f0e67ef876679f89
Aug 09 13:09:44 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280983, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:44.025543 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.006940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:37851 (local address 127.10.181.190:33161)
0809 13:09:44.007402 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:44.007408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:44.008546 (+ 1138us) server_negotiation.cc:407] Connection header received
0809 13:09:44.010783 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:44.010787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:44.010849 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:44.010935 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:44.012864 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.013645 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.015641 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.016024 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.016115 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.019835 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:44.019840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:44.019843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:44.019876 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:44.022600 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.023288 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.023294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.023297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.023420 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.023825 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.023830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.023833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.024095 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:44.024224 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:44.024907 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 13:09:44.025134 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":235,"threads_started":1}
I20260809 13:09:44.026468 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.007524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33161 (local address 127.10.181.129:37851)
0809 13:09:44.008362 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:44.008416 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:44.010511 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:44.011205 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:44.011219 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:44.011906 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:44.012683 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.012709 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.013797 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.013803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.015510 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.015524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.016242 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.016249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.016512 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.017191 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:44.017225 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:44.019631 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:44.022733 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.022742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.022756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.023174 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.023535 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.023540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.023544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.023716 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.024219 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:44.024227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:44.024551 (+ 324us) client_negotiation.cc:769] Sending connection context
0809 13:09:44.024812 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 13:09:44.025309 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260809 13:09:44.027963 17903 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33161
I20260809 13:09:44.028291 17903 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:44.029093 17903 heartbeater.cc:511] Master 127.10.181.190:33161 requested a full tablet report, sending...
I20260809 13:09:44.031564 17570 ts_manager.cc:194] Registered new tserver with Master: b9cb45fde49e433990acdd0c897eaf9b (127.10.181.129:44495)
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
I20260809 13:09:44.033334 17570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:37851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:44.059567 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:44.378705 17920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:44.378984 17920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:44.379127 17920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:44.386145 17920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:44.386260 17920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:44.386413 17920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:44.395605 17920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:44.396811 17920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:44.398097 17920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:44.410120 17926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:44.410177 17932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:44.410296 17927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:44.411180 17920 server_base.cc:1034] running on GCE node
I20260809 13:09:44.411751 17920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:44.412647 17920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:44.413852 17920 hybrid_clock.cc:648] HybridClock initialized: now 1786280984413800 us; error 66 us; skew 500 ppm
I20260809 13:09:44.414155 17920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:44 dist-test-slave-42z9 krb5kdc[17497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280984, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:44.422883 17920 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:44.424014 17920 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:44.424118 17920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:44.424414 17920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:44.427290 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "c84cad5db3664f6bbfe28e0395ce5fc9"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "178edbe457f8dd5306df43e1ccb78b64"
server_key_iv: "9965e1067cfb2828bcafeb97ac8418e6"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.428064 17920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "c84cad5db3664f6bbfe28e0395ce5fc9"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "178edbe457f8dd5306df43e1ccb78b64"
server_key_iv: "9965e1067cfb2828bcafeb97ac8418e6"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.432943 17920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:09:44.436335 17942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.437613 17920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:44.437763 17920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "c84cad5db3664f6bbfe28e0395ce5fc9"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "178edbe457f8dd5306df43e1ccb78b64"
server_key_iv: "9965e1067cfb2828bcafeb97ac8418e6"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.437954 17920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:44.454797 17920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:44.458544 17920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:44.458796 17920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:44.459744 17920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:44.460994 17920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:44.461088 17920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.461191 17920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:44.461259 17920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.490958 17920 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:45569
I20260809 13:09:44.491017 18134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:45569 every 8 connection(s)
I20260809 13:09:44.491743 17920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:44.497298 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 17920
I20260809 13:09:44.497531 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:44.498111 10966 external_mini_cluster.cc:1428] Setting key 3da4f1ce7dd2f7792cf569cbe69da14e
Aug 09 13:09:44 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280984, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:44.511626 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.494164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:60629 (local address 127.10.181.190:33161)
0809 13:09:44.494333 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:44.494338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:44.495375 (+ 1037us) server_negotiation.cc:407] Connection header received
0809 13:09:44.497370 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:44.497376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:44.497490 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:44.497587 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:44.499567 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.500537 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.502101 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.502497 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.502608 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.506173 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:44.506178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:44.506182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:44.506213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:44.508517 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.509314 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.509319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.509322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.509498 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.509873 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.509877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.509880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.510212 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:44.510362 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:44.511068 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 13:09:44.511292 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:09:44.512284 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.494484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33161 (local address 127.10.181.130:60629)
0809 13:09:44.495188 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:44.495226 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:44.497059 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:44.497849 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:44.497860 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:44.498624 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:44.499351 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.499406 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.500688 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.500694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.501956 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.501969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.502712 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.502720 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.502920 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.503575 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:44.503598 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:44.505893 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:44.508653 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.508662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.508676 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.509207 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.509607 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.509611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.509613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.509781 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.510316 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:44.510322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:44.510675 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 13:09:44.510933 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 13:09:44.511290 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":275,"threads_started":1}
I20260809 13:09:44.513599 18136 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33161
I20260809 13:09:44.513921 18136 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:44.514722 18136 heartbeater.cc:511] Master 127.10.181.190:33161 requested a full tablet report, sending...
I20260809 13:09:44.516206 17570 ts_manager.cc:194] Registered new tserver with Master: c84cad5db3664f6bbfe28e0395ce5fc9 (127.10.181.130:45569)
I20260809 13:09:44.516996 17570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:60629
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:44.549711 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:44.873333 18143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:44.873592 18143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:44.873734 18143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:44.880594 18143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:44.880697 18143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:44.880852 18143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:44.890062 18143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:44.891203 18143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:44.892410 18143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:44.904523 18153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:44.904498 18151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:44.904625 18149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:44.905207 18143 server_base.cc:1034] running on GCE node
I20260809 13:09:44.905805 18143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:44.906646 18143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:44.907829 18143 hybrid_clock.cc:648] HybridClock initialized: now 1786280984907778 us; error 47 us; skew 500 ppm
I20260809 13:09:44.908111 18143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:44 dist-test-slave-42z9 krb5kdc[17497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280984, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:44.911046 18143 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:44.912124 18143 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:44.912225 18143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:44.912523 18143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:44.915330 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "a4343887c2fb4fafa9f0719265380377"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "e0c382cbc976487d167368789de8afca"
server_key_iv: "54e7421a9abd262787d74f9639a73039"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.916103 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "a4343887c2fb4fafa9f0719265380377"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "e0c382cbc976487d167368789de8afca"
server_key_iv: "54e7421a9abd262787d74f9639a73039"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.921101 18143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:09:44.924433 18159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.925707 18143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 13:09:44.925848 18143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "a4343887c2fb4fafa9f0719265380377"
format_stamp: "Formatted at 2026-08-09 13:09:44 on dist-test-slave-42z9"
server_key: "e0c382cbc976487d167368789de8afca"
server_key_iv: "54e7421a9abd262787d74f9639a73039"
server_key_version: "encryptionkey@0"
I20260809 13:09:44.926038 18143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:44.937176 18143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:44.941120 18143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:44.941359 18143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:44.942327 18143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:44.943608 18143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:44.943706 18143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.943807 18143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:44.943872 18143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:44.972045 18143 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:42765
I20260809 13:09:44.972082 18346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:42765 every 8 connection(s)
I20260809 13:09:44.972852 18143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:44.979944 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 18143
I20260809 13:09:44.980134 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:44.980538 10966 external_mini_cluster.cc:1428] Setting key cae9a8e1e35c62573c594252b7c285e0
Aug 09 13:09:44 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280984, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:44.991343 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.975178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:45673 (local address 127.10.181.190:33161)
0809 13:09:44.975380 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:44.975387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:44.976511 (+ 1124us) server_negotiation.cc:407] Connection header received
0809 13:09:44.978239 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:44.978244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:44.978348 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:44.978431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:44.979842 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.980681 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.982477 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.982892 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.983001 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.986181 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:44.986186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:44.986189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:44.986221 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:44.988399 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.989091 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.989095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.989097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.989228 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:44.989674 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:44.989678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:44.989680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:44.989971 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:44.990113 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:44.990777 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 13:09:44.990992 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:09:44.992192 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:44.975684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33161 (local address 127.10.181.131:45673)
0809 13:09:44.976364 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:44.976400 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:44.977985 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:44.978617 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:44.978624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:44.979114 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:44.979669 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.979687 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.980828 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.980833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.982323 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:44.982336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:44.983113 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:44.983121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:44.983342 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:44.983941 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:44.983967 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:44.985978 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:44.988528 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.988537 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.988550 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.988954 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.989333 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:44.989339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:44.989342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:44.989554 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:44.990100 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:44.990107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:44.990415 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 13:09:44.990651 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 13:09:44.991111 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":269,"threads_started":1}
I20260809 13:09:44.993379 18347 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33161
I20260809 13:09:44.993690 18347 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:44.994465 18347 heartbeater.cc:511] Master 127.10.181.190:33161 requested a full tablet report, sending...
I20260809 13:09:44.996001 17570 ts_manager.cc:194] Registered new tserver with Master: a4343887c2fb4fafa9f0719265380377 (127.10.181.131:42765)
I20260809 13:09:44.996769 17570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:45673
I20260809 13:09:45.007494 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:45.023032 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.011882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.10.181.190:33161)
0809 13:09:45.012032 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.012037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.012540 (+ 503us) server_negotiation.cc:407] Connection header received
0809 13:09:45.012676 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.012680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.012734 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.012811 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:45.014217 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.014898 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.016021 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.016409 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.016500 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.018574 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:45.018579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:45.018582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:45.018612 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:45.021003 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.021549 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.021552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.021555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.021682 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.021952 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.021955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.021956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.022249 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:45.022437 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.022663 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.022812 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:09:45.026679 17570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:45.033258 17570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:45.036306 17903 heartbeater.cc:503] Master 127.10.181.190:33161 was elected leader, sending a full tablet report...
I20260809 13:09:45.056604 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.047730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:42765 (local address 127.0.0.1:55082)
0809 13:09:45.049042 (+ 1312us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:45.049058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:45.049194 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:45.050002 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:45.050007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:45.050032 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:45.050410 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.050422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.052140 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.052145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.053919 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.053934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.055578 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.055584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.055812 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.055851 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 13:09:45.055968 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 13:09:45.056165 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"mutex_wait_us":25,"thread_start_us":368,"threads_started":1}
I20260809 13:09:45.056744 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.045472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.10.181.130:45569)
0809 13:09:45.048729 (+ 3257us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.048737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.048766 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:09:45.048876 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.048883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.049122 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.049326 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:45.050019 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.051587 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.054413 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.055817 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.055980 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.056149 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.056376 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.056490 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3106,"thread_start_us":2996,"threads_started":1}
I20260809 13:09:45.057300 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.045229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:45569 (local address 127.0.0.1:50232)
0809 13:09:45.045930 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:45.045978 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:45.046154 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:45.049332 (+ 3178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:45.049339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:45.049370 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:45.049808 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.049825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.051732 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.051736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.054256 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.054266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.056040 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.056046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.056198 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.056228 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:09:45.056319 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:09:45.056432 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":92,"thread_start_us":774,"threads_started":1}
I20260809 13:09:45.057340 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.047193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.10.181.131:42765)
0809 13:09:45.047949 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.047957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.049166 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 13:09:45.049425 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.049432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.049643 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.049759 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:45.050648 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.052005 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.054103 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.055380 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.055558 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.055751 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.056864 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.057021 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":663,"threads_started":1}
I20260809 13:09:45.058914 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.045747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:44495 (local address 127.0.0.1:47896)
0809 13:09:45.048380 (+ 2633us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:45.048397 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:45.048612 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:45.049469 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:45.049474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:45.049500 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:45.049921 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.049931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.051548 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.051554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.054231 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.054247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.057279 (+ 3032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.057286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.058281 (+ 995us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.058317 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:09:45.058408 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:09:45.058526 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2496,"mutex_wait_us":133,"thread_start_us":3036,"threads_started":1}
I20260809 13:09:45.058945 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.046313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47896 (local address 127.10.181.129:44495)
0809 13:09:45.046931 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.046940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.048496 (+ 1556us) server_negotiation.cc:407] Connection header received
0809 13:09:45.048804 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.048812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.049038 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.049181 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:45.050059 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.051427 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.054413 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.055817 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.056016 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.056149 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.058456 (+ 2307us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.058631 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":299,"threads_started":1}
I20260809 13:09:45.060721 18013 tablet_service.cc:1467] Processing CreateTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf (DEFAULT_TABLE table=test-table [id=68b0749d778c4d199163f70ca70b7cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:45.061084 18228 tablet_service.cc:1467] Processing CreateTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf (DEFAULT_TABLE table=test-table [id=68b0749d778c4d199163f70ca70b7cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:45.061960 18013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a35ba640fed4c538da0cbe3017ceabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:45.062343 18228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a35ba640fed4c538da0cbe3017ceabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:45.072937 18370 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Bootstrap starting.
I20260809 13:09:45.072654 17781 tablet_service.cc:1467] Processing CreateTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf (DEFAULT_TABLE table=test-table [id=68b0749d778c4d199163f70ca70b7cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:45.073958 18371 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Bootstrap starting.
I20260809 13:09:45.074123 17781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a35ba640fed4c538da0cbe3017ceabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:45.077983 18371 tablet_bootstrap.cc:654] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:45.079274 18371 log.cc:824] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:45.079828 18370 tablet_bootstrap.cc:654] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:45.080963 18370 log.cc:824] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:45.082231 18371 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: No bootstrap required, opened a new log
I20260809 13:09:45.082543 18371 ts_tablet_manager.cc:1397] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 13:09:45.083940 18370 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: No bootstrap required, opened a new log
I20260809 13:09:45.084183 18370 ts_tablet_manager.cc:1397] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260809 13:09:45.090135 18374 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Bootstrap starting.
I20260809 13:09:45.092743 18374 tablet_bootstrap.cc:654] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:45.092519 18371 raft_consensus.cc:348] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.092983 18371 raft_consensus.cc:374] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:45.093101 18371 raft_consensus.cc:729] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84cad5db3664f6bbfe28e0395ce5fc9, State: Initialized, Role: FOLLOWER
I20260809 13:09:45.093818 18374 log.cc:824] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:45.093720 18371 consensus_queue.cc:260] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.094436 18370 raft_consensus.cc:348] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.094888 18370 raft_consensus.cc:374] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:45.094913 18136 heartbeater.cc:503] Master 127.10.181.190:33161 was elected leader, sending a full tablet report...
I20260809 13:09:45.095057 18370 raft_consensus.cc:729] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4343887c2fb4fafa9f0719265380377, State: Initialized, Role: FOLLOWER
I20260809 13:09:45.095541 18371 ts_tablet_manager.cc:1428] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260809 13:09:45.095768 18370 consensus_queue.cc:260] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.096890 18347 heartbeater.cc:503] Master 127.10.181.190:33161 was elected leader, sending a full tablet report...
I20260809 13:09:45.098449 18374 tablet_bootstrap.cc:492] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: No bootstrap required, opened a new log
I20260809 13:09:45.098742 18374 ts_tablet_manager.cc:1397] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260809 13:09:45.099846 18370 ts_tablet_manager.cc:1428] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260809 13:09:45.105881 18374 raft_consensus.cc:348] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.106177 18374 raft_consensus.cc:374] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:45.106242 18374 raft_consensus.cc:729] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9cb45fde49e433990acdd0c897eaf9b, State: Initialized, Role: FOLLOWER
I20260809 13:09:45.106693 18374 consensus_queue.cc:260] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.107856 18374 ts_tablet_manager.cc:1428] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20260809 13:09:45.126893 18375 raft_consensus.cc:482] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:45.127141 18375 raft_consensus.cc:504] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.128613 18375 leader_election.cc:288] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4343887c2fb4fafa9f0719265380377 (127.10.181.131:42765), b9cb45fde49e433990acdd0c897eaf9b (127.10.181.129:44495)
I20260809 13:09:45.137472 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.129182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:54537 (local address 127.10.181.129:44495)
0809 13:09:45.129345 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.129349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.130203 (+ 854us) server_negotiation.cc:407] Connection header received
0809 13:09:45.130355 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.130360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.130420 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.130499 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:45.131278 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.132399 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.135185 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.136365 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.136516 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.136590 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.137086 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.137214 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:09:45.137470 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.129042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:54065 (local address 127.10.181.131:42765)
0809 13:09:45.129240 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.129247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.130112 (+ 865us) server_negotiation.cc:407] Connection header received
0809 13:09:45.130323 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.130329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.130404 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.130500 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:09:45.131326 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.132400 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.135214 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.136366 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.136516 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.136598 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.137085 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.137214 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:09:45.137683 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.129020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:44495 (local address 127.10.181.130:54537)
0809 13:09:45.130110 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:45.130125 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:45.130241 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:45.130522 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:45.130527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:45.130568 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:45.131101 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.131116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.132519 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.132523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.135017 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.135034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.136637 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.136643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.136868 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.136950 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 13:09:45.137041 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:09:45.137189 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":302,"spinlock_wait_cycles":38400,"thread_start_us":431,"threads_started":1}
I20260809 13:09:45.137957 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.129020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:42765 (local address 127.10.181.130:54065)
0809 13:09:45.130000 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:45.130020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:45.130194 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:45.130522 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:45.130527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:45.130570 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:09:45.131199 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.131211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.132514 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.132518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.135026 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:45.135035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.136610 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.136620 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:45.136866 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.136949 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 13:09:45.137038 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:09:45.137189 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":209,"spinlock_wait_cycles":18432,"thread_start_us":744,"threads_started":1}
I20260809 13:09:45.138871 17820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a35ba640fed4c538da0cbe3017ceabf" candidate_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9cb45fde49e433990acdd0c897eaf9b" is_pre_election: true
I20260809 13:09:45.138900 18270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a35ba640fed4c538da0cbe3017ceabf" candidate_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4343887c2fb4fafa9f0719265380377" is_pre_election: true
I20260809 13:09:45.139323 18270 raft_consensus.cc:2454] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84cad5db3664f6bbfe28e0395ce5fc9 in term 0.
I20260809 13:09:45.139309 17820 raft_consensus.cc:2454] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84cad5db3664f6bbfe28e0395ce5fc9 in term 0.
I20260809 13:09:45.139943 17948 leader_election.cc:302] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4343887c2fb4fafa9f0719265380377, c84cad5db3664f6bbfe28e0395ce5fc9; no voters:
I20260809 13:09:45.140293 18375 raft_consensus.cc:2781] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:45.140417 18375 raft_consensus.cc:482] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:45.140522 18375 raft_consensus.cc:3037] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:45.143090 18375 raft_consensus.cc:504] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.143721 18375 leader_election.cc:288] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [CANDIDATE]: Term 1 election: Requested vote from peers a4343887c2fb4fafa9f0719265380377 (127.10.181.131:42765), b9cb45fde49e433990acdd0c897eaf9b (127.10.181.129:44495)
I20260809 13:09:45.144078 18270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a35ba640fed4c538da0cbe3017ceabf" candidate_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4343887c2fb4fafa9f0719265380377"
I20260809 13:09:45.144156 17820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a35ba640fed4c538da0cbe3017ceabf" candidate_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9cb45fde49e433990acdd0c897eaf9b"
I20260809 13:09:45.144275 18270 raft_consensus.cc:3037] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:45.144323 17820 raft_consensus.cc:3037] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:45.147346 18270 raft_consensus.cc:2454] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84cad5db3664f6bbfe28e0395ce5fc9 in term 1.
I20260809 13:09:45.147346 17820 raft_consensus.cc:2454] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84cad5db3664f6bbfe28e0395ce5fc9 in term 1.
I20260809 13:09:45.147734 17948 leader_election.cc:302] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a4343887c2fb4fafa9f0719265380377, b9cb45fde49e433990acdd0c897eaf9b, c84cad5db3664f6bbfe28e0395ce5fc9; no voters:
I20260809 13:09:45.148033 18375 raft_consensus.cc:2781] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:45.148429 18375 raft_consensus.cc:686] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 1 LEADER]: Becoming Leader. State: Replica: c84cad5db3664f6bbfe28e0395ce5fc9, State: Running, Role: LEADER
I20260809 13:09:45.148844 18375 consensus_queue.cc:237] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } }
I20260809 13:09:45.156586 17568 catalog_manager.cc:5495] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c84cad5db3664f6bbfe28e0395ce5fc9 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84cad5db3664f6bbfe28e0395ce5fc9" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 } health_report { overall_health: UNKNOWN } } }
I20260809 13:09:45.177981 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.173180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.10.181.130:45569)
0809 13:09:45.173381 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.173387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.173495 (+ 108us) server_negotiation.cc:407] Connection header received
0809 13:09:45.173674 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.173678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.173762 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.173875 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:09:45.174158 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.174877 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.176098 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.176494 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.176583 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.176981 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:09:45.177498 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:09:45.177622 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.177669 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.177733 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:09:45.185722 18270 raft_consensus.cc:1261] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 1 FOLLOWER]: Refusing update from remote peer c84cad5db3664f6bbfe28e0395ce5fc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:45.185714 17820 raft_consensus.cc:1261] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 1 FOLLOWER]: Refusing update from remote peer c84cad5db3664f6bbfe28e0395ce5fc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:45.186720 18385 consensus_queue.cc:1035] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9cb45fde49e433990acdd0c897eaf9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:45.187042 18375 consensus_queue.cc:1035] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4343887c2fb4fafa9f0719265380377" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 42765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:45.198536 18390 mvcc.cc:205] Tried to move back new op lower bound from 7316606915310333952 to 7316606915173330944. Current Snapshot: MvccSnapshot[applied={T|T < 7316606915310333952}]
I20260809 13:09:45.201305 18391 mvcc.cc:205] Tried to move back new op lower bound from 7316606915310333952 to 7316606915173330944. Current Snapshot: MvccSnapshot[applied={T|T < 7316606915310333952}]
I20260809 13:09:45.211879 17568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:45.212137 17568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:45.215389 17568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1908818a653f4c34940fe18321f8dcfa: Sending DeleteTablet for 3 replicas of tablet 9a35ba640fed4c538da0cbe3017ceabf
I20260809 13:09:45.216529 18228 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:45 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:55082
I20260809 13:09:45.216490 18013 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:45 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:50232
I20260809 13:09:45.217186 17781 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a35ba640fed4c538da0cbe3017ceabf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:45 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:47896
I20260809 13:09:45.218080 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 17676
I20260809 13:09:45.220965 18406 tablet_replica.cc:330] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: stopping tablet replica
I20260809 13:09:45.221416 18406 raft_consensus.cc:2229] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:45.222060 18406 raft_consensus.cc:2258] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:45.222870 18405 tablet_replica.cc:330] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: stopping tablet replica
I20260809 13:09:45.223320 18405 raft_consensus.cc:2229] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:45.224082 18405 raft_consensus.cc:2258] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:45.224954 18410 tablet_replica.cc:330] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: stopping tablet replica
I20260809 13:09:45.225343 18410 raft_consensus.cc:2229] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:45.225694 18406 ts_tablet_manager.cc:1905] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:45.225813 18410 raft_consensus.cc:2258] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:45.226069 18405 ts_tablet_manager.cc:1905] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:45.227802 18410 ts_tablet_manager.cc:1905] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:45.232254 18406 ts_tablet_manager.cc:1918] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:45.232416 18406 log.cc:1196] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/9a35ba640fed4c538da0cbe3017ceabf
I20260809 13:09:45.232944 18406 ts_tablet_manager.cc:1939] T 9a35ba640fed4c538da0cbe3017ceabf P a4343887c2fb4fafa9f0719265380377: Deleting consensus metadata
I20260809 13:09:45.234918 17548 catalog_manager.cc:4841] TS a4343887c2fb4fafa9f0719265380377 (127.10.181.131:42765): tablet 9a35ba640fed4c538da0cbe3017ceabf (table test-table [id=68b0749d778c4d199163f70ca70b7cef]) successfully deleted
I20260809 13:09:45.235584 18410 ts_tablet_manager.cc:1918] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:45.235731 18410 log.cc:1196] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/9a35ba640fed4c538da0cbe3017ceabf
I20260809 13:09:45.236129 18410 ts_tablet_manager.cc:1939] T 9a35ba640fed4c538da0cbe3017ceabf P b9cb45fde49e433990acdd0c897eaf9b: Deleting consensus metadata
I20260809 13:09:45.233825 18405 ts_tablet_manager.cc:1918] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:45.236470 18405 log.cc:1196] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/9a35ba640fed4c538da0cbe3017ceabf
Aug 09 13:09:45 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:45.237129 18405 ts_tablet_manager.cc:1939] T 9a35ba640fed4c538da0cbe3017ceabf P c84cad5db3664f6bbfe28e0395ce5fc9: Deleting consensus metadata
I20260809 13:09:45.237780 17546 catalog_manager.cc:4841] TS b9cb45fde49e433990acdd0c897eaf9b (127.10.181.129:44495): tablet 9a35ba640fed4c538da0cbe3017ceabf (table test-table [id=68b0749d778c4d199163f70ca70b7cef]) successfully deleted
I20260809 13:09:45.238835 17548 catalog_manager.cc:4841] TS c84cad5db3664f6bbfe28e0395ce5fc9 (127.10.181.130:45569): tablet 9a35ba640fed4c538da0cbe3017ceabf (table test-table [id=68b0749d778c4d199163f70ca70b7cef]) successfully deleted
I20260809 13:09:45.243871 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.219203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.10.181.129:44495)
0809 13:09:45.219366 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.219372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.226511 (+ 7139us) server_negotiation.cc:407] Connection header received
0809 13:09:45.226806 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.226811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.226898 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.226967 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:45.228636 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.229855 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.231595 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.232273 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.232374 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.238220 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:45.238234 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:45.238251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:45.238287 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:45.241096 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.241893 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.241902 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.241906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.242052 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.242447 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.242451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.242453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.242859 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:45.243092 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.243440 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.243620 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:45.244532 17896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:45.332548 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 17676
I20260809 13:09:45.366171 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 17920
Aug 09 13:09:45 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:45.382779 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.366996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.10.181.130:45569)
0809 13:09:45.367175 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.367179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.367198 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:09:45.367411 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.367415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.367503 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.367591 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:45.369092 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.369951 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.371702 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.372132 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.372224 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.377035 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:45.377050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:45.377067 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:45.377108 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:45.380062 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.380817 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.380825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.380827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.380968 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.381349 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.381352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.381354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.381763 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:45.382013 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.382309 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.382471 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:09:45.383405 18130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:45.495193 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 17920
I20260809 13:09:45.519488 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 18143
Aug 09 13:09:45 dist-test-slave-42z9 krb5kdc[17497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:45.533608 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:45.520209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.10.181.131:42765)
0809 13:09:45.520368 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:45.520373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:45.520393 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:09:45.520587 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:45.520591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:45.520661 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:45.520757 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:45.522108 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.522882 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.524074 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:45.524462 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:45.524554 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:45.528611 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:45.528630 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:45.528648 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:45.528682 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:45.531423 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.531981 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.531988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.531990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.532126 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:45.532400 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:45.532403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:45.532405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:45.532737 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:45.532968 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:45.533145 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 13:09:45.533319 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:45.534240 18337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:45.621380 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 18143
I20260809 13:09:45.644829 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 17517
I20260809 13:09:45.645998 17620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:45.754817 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 17517
2026-08-09T13:09:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4172 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T13:09:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:45Z Disabled control of system clock
I20260809 13:09:45.834888 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34453
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 13:09:46.153335 18422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:46.153612 18422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:46.153702 18422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:46.160611 18422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:46.160701 18422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:46.160743 18422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:46.160797 18422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:46.160840 18422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:46.169407 18422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:46.170590 18422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:46.172224 18422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:46.183259 18431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:46.183317 18430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:46.183483 18434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:46.184049 18422 server_base.cc:1034] running on GCE node
I20260809 13:09:46.184748 18422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:46.186028 18422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:46.187237 18422 hybrid_clock.cc:648] HybridClock initialized: now 1786280986187192 us; error 32 us; skew 500 ppm
I20260809 13:09:46.187595 18422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:46.189688 18422 webserver.cc:460] Webserver started at http://127.10.181.190:34001/ using document root <none> and password file <none>
I20260809 13:09:46.190318 18422 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:46.190412 18422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:46.190748 18422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:46.193576 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "0377ac9476174990ba748b8e06c6f177"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "f859047ed5accabb0b5a3c4724aab3c5"
server_key_iv: "bc6916b984d6b6c1837fbcca6e00b089"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.194309 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "0377ac9476174990ba748b8e06c6f177"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "f859047ed5accabb0b5a3c4724aab3c5"
server_key_iv: "bc6916b984d6b6c1837fbcca6e00b089"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.199333 18422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:46.202648 18445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:46.203862 18422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:09:46.204007 18422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "0377ac9476174990ba748b8e06c6f177"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "f859047ed5accabb0b5a3c4724aab3c5"
server_key_iv: "bc6916b984d6b6c1837fbcca6e00b089"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.204208 18422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:46.209663 18422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:46.210341 18422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:46.210618 18422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:46.227497 18422 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34453
I20260809 13:09:46.227528 18523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34453 every 8 connection(s)
I20260809 13:09:46.228678 18422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:46.231513 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 18422
I20260809 13:09:46.231720 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:46.232522 10966 external_mini_cluster.cc:1428] Setting key d2732e54ff86e0912170166d0e8099ef
I20260809 13:09:46.232604 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:46.240676 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:46.234682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.10.181.190:34453)
0809 13:09:46.235796 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:46.235807 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:46.235854 (+ 47us) server_negotiation.cc:407] Connection header received
0809 13:09:46.237410 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:46.237462 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:46.237874 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:46.238187 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:46.239225 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:46.239231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:46.239246 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:46.239263 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:46.239316 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:46.239336 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:46.239521 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:46.239633 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 13:09:46.239870 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":372,"threads_started":1}
I20260809 13:09:46.245180 18524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: Bootstrap starting.
I20260809 13:09:46.247962 18524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:46.249054 18524 log.cc:824] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:46.251519 18524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: No bootstrap required, opened a new log
I20260809 13:09:46.258422 18524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } }
I20260809 13:09:46.258720 18524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:46.258791 18524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0377ac9476174990ba748b8e06c6f177, State: Initialized, Role: FOLLOWER
I20260809 13:09:46.259243 18524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } }
I20260809 13:09:46.259452 18524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:46.259548 18524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:46.259662 18524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:46.261619 18524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } }
I20260809 13:09:46.262034 18524 leader_election.cc:302] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0377ac9476174990ba748b8e06c6f177; no voters:
I20260809 13:09:46.262566 18524 leader_election.cc:288] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:46.262674 18536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:46.263752 18536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [term 1 LEADER]: Becoming Leader. State: Replica: 0377ac9476174990ba748b8e06c6f177, State: Running, Role: LEADER
I20260809 13:09:46.264216 18536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } }
I20260809 13:09:46.264496 18524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:46.266779 18538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0377ac9476174990ba748b8e06c6f177. Latest consensus state: current_term: 1 leader_uuid: "0377ac9476174990ba748b8e06c6f177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } } }
I20260809 13:09:46.266819 18537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0377ac9476174990ba748b8e06c6f177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0377ac9476174990ba748b8e06c6f177" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34453 } } }
I20260809 13:09:46.267159 18538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:46.267163 18537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:46.268049 18543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:46.272827 18543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:46.286623 18543 catalog_manager.cc:1347] Generated new cluster ID: 00d25ddd5f7c47cc8a890bc472ba09db
I20260809 13:09:46.286731 18543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:46.295800 18543 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:46.296277 18543 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:46.302029 18543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: Generated new TSK 0
I20260809 13:09:46.302685 18543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:09:46.310699 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--builtin_ntp_servers=127.10.181.148:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 13:09:46.638517 18561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:46.638774 18561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:46.638870 18561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:46.645896 18561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:46.645982 18561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:46.646095 18561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:46.654856 18561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:46.656028 18561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:46.657755 18561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:46.670548 18572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:46.670595 18575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:46.670639 18570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:46.671648 18561 server_base.cc:1034] running on GCE node
I20260809 13:09:46.672231 18561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:46.673121 18561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:46.674317 18561 hybrid_clock.cc:648] HybridClock initialized: now 1786280986674264 us; error 46 us; skew 500 ppm
I20260809 13:09:46.674623 18561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:46.676752 18561 webserver.cc:460] Webserver started at http://127.10.181.129:32783/ using document root <none> and password file <none>
I20260809 13:09:46.677466 18561 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:46.677580 18561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:46.677892 18561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:46.680678 18561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "64f4ffca2814417eb05319d1ba05c685"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "e61221b47bd23b03c514d71bd7d920f9"
server_key_iv: "8fcdba6958719807136c6034a5e34fe5"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.681478 18561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "64f4ffca2814417eb05319d1ba05c685"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "e61221b47bd23b03c514d71bd7d920f9"
server_key_iv: "8fcdba6958719807136c6034a5e34fe5"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.686456 18561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 13:09:46.689764 18592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:46.691000 18561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:09:46.691161 18561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "64f4ffca2814417eb05319d1ba05c685"
format_stamp: "Formatted at 2026-08-09 13:09:46 on dist-test-slave-42z9"
server_key: "e61221b47bd23b03c514d71bd7d920f9"
server_key_iv: "8fcdba6958719807136c6034a5e34fe5"
server_key_version: "encryptionkey@0"
I20260809 13:09:46.691359 18561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:46.696772 18561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:46.697605 18561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:46.697849 18561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:46.698752 18561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:46.700109 18561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:46.700205 18561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:46.700307 18561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:46.700376 18561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:46.730525 18561 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:41851
I20260809 13:09:46.730569 18784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:41851 every 8 connection(s)
I20260809 13:09:46.731750 18561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:46.732906 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 18561
I20260809 13:09:46.733107 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:46.733937 10966 external_mini_cluster.cc:1428] Setting key cc380b9e51f81129ef3efd31fdf30ad3
I20260809 13:09:46.736810 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--builtin_ntp_servers=127.10.181.148:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 13:09:46.740092 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:46.734474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:60097 (local address 127.10.181.190:34453)
0809 13:09:46.734657 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:46.734661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:46.736128 (+ 1467us) server_negotiation.cc:407] Connection header received
0809 13:09:46.737893 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:46.737899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:46.737964 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:46.738051 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:46.739353 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:46.739359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:46.739364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:46.739394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:46.739427 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:46.739439 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:46.739539 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:46.739736 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 13:09:46.739811 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:09:46.741029 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:46.734922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34453 (local address 127.10.181.129:60097)
0809 13:09:46.735968 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:46.736014 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:46.737638 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:46.738320 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:46.738332 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:46.738418 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:46.739052 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:46.739055 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:46.739093 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:46.739188 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:46.739529 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:46.739540 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:46.739548 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:09:46.739808 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 13:09:46.740045 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":302,"threads_started":1}
I20260809 13:09:46.746217 18785 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34453
I20260809 13:09:46.746490 18785 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:46.747049 18785 heartbeater.cc:511] Master 127.10.181.190:34453 requested a full tablet report, sending...
I20260809 13:09:46.749042 18476 ts_manager.cc:194] Registered new tserver with Master: 64f4ffca2814417eb05319d1ba05c685 (127.10.181.129:41851)
W20260809 13:09:47.059304 18788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:47.059615 18788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:47.059756 18788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:47.066707 18788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:47.066802 18788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:47.066941 18788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:47.075560 18788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:47.076738 18788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:47.078522 18788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:47.091869 18798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:47.091831 18801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:47.091917 18799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:47.092859 18788 server_base.cc:1034] running on GCE node
I20260809 13:09:47.093406 18788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:47.094328 18788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:47.095520 18788 hybrid_clock.cc:648] HybridClock initialized: now 1786280987095464 us; error 51 us; skew 500 ppm
I20260809 13:09:47.095824 18788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:47.097915 18788 webserver.cc:460] Webserver started at http://127.10.181.130:46483/ using document root <none> and password file <none>
I20260809 13:09:47.098661 18788 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:47.098773 18788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:47.099100 18788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:47.101929 18788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "b7c28268516944a39bd72b9d08063f79"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "75ad16aab02d62c56a9cdebf24fc2887"
server_key_iv: "ede01228589f135a83f155e3a531eaf2"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.102686 18788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "b7c28268516944a39bd72b9d08063f79"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "75ad16aab02d62c56a9cdebf24fc2887"
server_key_iv: "ede01228589f135a83f155e3a531eaf2"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.107589 18788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:47.110816 18810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.112020 18788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 13:09:47.112169 18788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "b7c28268516944a39bd72b9d08063f79"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "75ad16aab02d62c56a9cdebf24fc2887"
server_key_iv: "ede01228589f135a83f155e3a531eaf2"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.112355 18788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:47.117623 18788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:47.118335 18788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:47.118569 18788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:47.119436 18788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:47.120733 18788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:47.120826 18788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.120932 18788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:47.121013 18788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.153580 18788 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:45051
I20260809 13:09:47.153657 18990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:45051 every 8 connection(s)
I20260809 13:09:47.155108 18788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:47.159343 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 18788
I20260809 13:09:47.159526 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:47.160092 10966 external_mini_cluster.cc:1428] Setting key 5f873c809a0748ef40b6f4950ed602ad
I20260809 13:09:47.163180 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--builtin_ntp_servers=127.10.181.148:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 13:09:47.163493 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.157623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:34031 (local address 127.10.181.190:34453)
0809 13:09:47.157796 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.157800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.159092 (+ 1292us) server_negotiation.cc:407] Connection header received
0809 13:09:47.161232 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.161236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.161289 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.161372 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.162748 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.162752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.162755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.162771 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.162803 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.162815 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.162915 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.163129 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.163195 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:47.164348 18993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.158075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34453 (local address 127.10.181.130:34031)
0809 13:09:47.158928 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.158970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.160970 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.161657 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.161668 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.161780 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.162453 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.162456 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.162498 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.162585 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.162901 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.162912 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.162920 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.163195 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.163436 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":257,"threads_started":1}
I20260809 13:09:47.165720 18991 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34453
I20260809 13:09:47.166008 18991 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:47.166564 18991 heartbeater.cc:511] Master 127.10.181.190:34453 requested a full tablet report, sending...
I20260809 13:09:47.167956 18476 ts_manager.cc:194] Registered new tserver with Master: b7c28268516944a39bd72b9d08063f79 (127.10.181.130:45051)
W20260809 13:09:47.491041 18994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:47.491273 18994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:47.491392 18994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:47.498301 18994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:47.498385 18994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:47.498519 18994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:47.507088 18994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:47.508216 18994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:47.509941 18994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:47.522122 19008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:47.522219 19003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:47.522184 19004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:47.523075 18994 server_base.cc:1034] running on GCE node
I20260809 13:09:47.523615 18994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:47.524466 18994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:47.525656 18994 hybrid_clock.cc:648] HybridClock initialized: now 1786280987525611 us; error 31 us; skew 500 ppm
I20260809 13:09:47.525946 18994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:47.527963 18994 webserver.cc:460] Webserver started at http://127.10.181.131:45611/ using document root <none> and password file <none>
I20260809 13:09:47.528594 18994 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:47.528692 18994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:47.529006 18994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:47.531810 18994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "74aa52a77d254649a6e0d45cc476a36c"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "1d308780576fba4f04ff62a93b615a7d"
server_key_iv: "df166608fcdf9570629541b4e35056dc"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.532565 18994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "74aa52a77d254649a6e0d45cc476a36c"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "1d308780576fba4f04ff62a93b615a7d"
server_key_iv: "df166608fcdf9570629541b4e35056dc"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.537348 18994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 13:09:47.540573 19018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.541782 18994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 13:09:47.541927 18994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "74aa52a77d254649a6e0d45cc476a36c"
format_stamp: "Formatted at 2026-08-09 13:09:47 on dist-test-slave-42z9"
server_key: "1d308780576fba4f04ff62a93b615a7d"
server_key_iv: "df166608fcdf9570629541b4e35056dc"
server_key_version: "encryptionkey@0"
I20260809 13:09:47.542128 18994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:47.547287 18994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:47.548000 18994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:47.548254 18994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:47.549080 18994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:47.550432 18994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:47.550531 18994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.550638 18994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:47.550709 18994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:47.579317 18994 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41347
I20260809 13:09:47.579366 19214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41347 every 8 connection(s)
I20260809 13:09:47.580513 18994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:47.585615 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 18994
I20260809 13:09:47.585776 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:47.586324 10966 external_mini_cluster.cc:1428] Setting key 371aadaa7d4590652ed54883114b7057
I20260809 13:09:47.588406 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.582860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:44751 (local address 127.10.181.190:34453)
0809 13:09:47.583041 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.583045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.584344 (+ 1299us) server_negotiation.cc:407] Connection header received
0809 13:09:47.586255 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.586259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.586316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.586401 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.587713 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.587718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.587721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.587737 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.587770 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.587783 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.587898 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.588103 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.588181 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:09:47.589251 19218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.583360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34453 (local address 127.10.181.131:44751)
0809 13:09:47.584209 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.584242 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.585979 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.586710 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.586721 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.586830 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.587436 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.587439 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.587474 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.587555 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.587877 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.587888 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.587896 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.588142 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.588369 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":255,"threads_started":1}
I20260809 13:09:47.590456 19215 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34453
I20260809 13:09:47.590718 19215 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:47.591264 19215 heartbeater.cc:511] Master 127.10.181.190:34453 requested a full tablet report, sending...
I20260809 13:09:47.592518 18476 ts_manager.cc:194] Registered new tserver with Master: 74aa52a77d254649a6e0d45cc476a36c (127.10.181.131:41347)
I20260809 13:09:47.601353 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:47.608040 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.605546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.10.181.190:34453)
0809 13:09:47.605683 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.605687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.606221 (+ 534us) server_negotiation.cc:407] Connection header received
0809 13:09:47.606392 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.606397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.606448 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.606526 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.607409 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.607415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.607420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.607438 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.607469 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.607480 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.607583 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.607718 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.607779 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 13:09:47.611680 18476 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:09:47.618450 18476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:09:47.643400 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.629836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:45051 (local address 127.0.0.1:58160)
0809 13:09:47.631110 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.631132 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.631227 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.641179 (+ 9952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.641187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.641284 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.642066 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.642070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.642101 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.642189 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.642682 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.642693 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.642701 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.642839 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.642933 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"thread_start_us":283,"threads_started":1}
I20260809 13:09:47.644385 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.631548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50750 (local address 127.10.181.129:41851)
0809 13:09:47.641075 (+ 9527us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.641083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.641111 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:09:47.641202 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.641209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.641406 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.642094 (+ 688us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.643474 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.643482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.643495 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.643520 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.643572 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.643591 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.643864 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.643945 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.644050 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9349,"thread_start_us":9220,"threads_started":1}
I20260809 13:09:47.644410 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.631210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50750)
0809 13:09:47.639605 (+ 8395us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.639629 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.639732 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.641656 (+ 1924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.641660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.641684 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.642241 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.642244 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.642260 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.642305 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.643815 (+ 1510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.643820 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.643825 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.643966 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.644023 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8273,"thread_start_us":8599,"threads_started":1}
I20260809 13:09:47.644531 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.629983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.10.181.130:45051)
0809 13:09:47.640582 (+ 10599us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.640601 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.640646 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:09:47.640726 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.640735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.640971 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.641144 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.642412 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.642424 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.642441 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.642469 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.642531 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.642550 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.644025 (+ 1475us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.644108 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.644201 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10417,"thread_start_us":12183,"threads_started":1}
I20260809 13:09:47.647289 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.629364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41347 (local address 127.0.0.1:59768)
0809 13:09:47.630515 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.630560 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.630693 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.643348 (+ 12655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.643354 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.643377 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.644076 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.644079 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.644094 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.644139 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.646311 (+ 2172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.646316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.646321 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.646601 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.646646 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":1439,"threads_started":1}
I20260809 13:09:47.648001 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.629599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.10.181.131:41347)
0809 13:09:47.642831 (+ 13232us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.642839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.642871 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:09:47.642937 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.642945 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.643145 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.644300 (+ 1155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.646095 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.646101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.646109 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.646122 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.646165 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.646182 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.647630 (+ 1448us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.647685 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.647753 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13060,"thread_start_us":12892,"threads_started":1}
I20260809 13:09:47.648322 18669 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2160ad4b5448580e3f25151ab6292 (DEFAULT_TABLE table=test-table [id=674b7bd05bda4d97b0aa66bc9a95d864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:47.648322 18881 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2160ad4b5448580e3f25151ab6292 (DEFAULT_TABLE table=test-table [id=674b7bd05bda4d97b0aa66bc9a95d864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:47.649878 18881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2160ad4b5448580e3f25151ab6292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:47.649878 18669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2160ad4b5448580e3f25151ab6292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:47.650866 19102 tablet_service.cc:1467] Processing CreateTablet for tablet 37d2160ad4b5448580e3f25151ab6292 (DEFAULT_TABLE table=test-table [id=674b7bd05bda4d97b0aa66bc9a95d864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:09:47.652173 19102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37d2160ad4b5448580e3f25151ab6292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:47.663610 19245 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Bootstrap starting.
I20260809 13:09:47.666180 19246 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Bootstrap starting.
I20260809 13:09:47.667260 19245 tablet_bootstrap.cc:654] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:47.668529 19245 log.cc:824] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:47.669273 19247 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Bootstrap starting.
I20260809 13:09:47.669997 19246 tablet_bootstrap.cc:654] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:47.671051 19246 log.cc:824] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:47.672410 19247 tablet_bootstrap.cc:654] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:47.672528 19245 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: No bootstrap required, opened a new log
I20260809 13:09:47.672796 19245 ts_tablet_manager.cc:1397] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:09:47.673349 19247 log.cc:824] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:47.674026 19246 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: No bootstrap required, opened a new log
I20260809 13:09:47.674322 19246 ts_tablet_manager.cc:1397] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260809 13:09:47.675552 19247 tablet_bootstrap.cc:492] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: No bootstrap required, opened a new log
I20260809 13:09:47.675779 19247 ts_tablet_manager.cc:1397] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 13:09:47.682715 19247 raft_consensus.cc:348] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.682853 19245 raft_consensus.cc:348] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.683187 19247 raft_consensus.cc:374] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:47.683300 19245 raft_consensus.cc:374] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:47.683316 19247 raft_consensus.cc:729] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c28268516944a39bd72b9d08063f79, State: Initialized, Role: FOLLOWER
I20260809 13:09:47.683403 19245 raft_consensus.cc:729] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74aa52a77d254649a6e0d45cc476a36c, State: Initialized, Role: FOLLOWER
I20260809 13:09:47.683890 19247 consensus_queue.cc:260] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.684096 19245 consensus_queue.cc:260] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.684209 19246 raft_consensus.cc:348] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.684626 19246 raft_consensus.cc:374] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:47.684768 19246 raft_consensus.cc:729] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f4ffca2814417eb05319d1ba05c685, State: Initialized, Role: FOLLOWER
I20260809 13:09:47.684864 18991 heartbeater.cc:503] Master 127.10.181.190:34453 was elected leader, sending a full tablet report...
I20260809 13:09:47.685472 19246 consensus_queue.cc:260] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.686177 19215 heartbeater.cc:503] Master 127.10.181.190:34453 was elected leader, sending a full tablet report...
I20260809 13:09:47.687279 19245 ts_tablet_manager.cc:1428] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260809 13:09:47.688104 18785 heartbeater.cc:503] Master 127.10.181.190:34453 was elected leader, sending a full tablet report...
I20260809 13:09:47.688716 19246 ts_tablet_manager.cc:1428] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260809 13:09:47.690279 19247 ts_tablet_manager.cc:1428] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Time spent starting tablet: real 0.014s user 0.010s sys 0.000s
I20260809 13:09:47.700055 19254 raft_consensus.cc:482] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:09:47.700292 19254 raft_consensus.cc:504] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.701828 19254 leader_election.cc:288] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74aa52a77d254649a6e0d45cc476a36c (127.10.181.131:41347), 64f4ffca2814417eb05319d1ba05c685 (127.10.181.129:41851)
I20260809 13:09:47.704931 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.702132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:36425 (local address 127.10.181.131:41347)
0809 13:09:47.702307 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.702313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.702931 (+ 618us) server_negotiation.cc:407] Connection header received
0809 13:09:47.703066 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.703080 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.703142 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.703236 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.704392 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.704396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.704399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.704419 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.704460 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.704476 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.704596 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.704678 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.704746 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:09:47.705224 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.702184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41347 (local address 127.10.181.130:36425)
0809 13:09:47.702839 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.702855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.702964 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.703418 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.703423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.703451 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.704207 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.704212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.704228 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.704268 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.704586 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.704589 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.704593 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.704725 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.704780 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":339,"threads_started":1}
I20260809 13:09:47.705643 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.703184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59987 (local address 127.10.181.129:41851)
0809 13:09:47.703353 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.703359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.703616 (+ 257us) server_negotiation.cc:407] Connection header received
0809 13:09:47.703774 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.703778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.703824 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.703918 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.704799 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.704805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.704810 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.704828 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.704865 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.704881 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.704987 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.705233 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.705298 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:09:47.705731 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.702936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.10.181.130:59987)
0809 13:09:47.703520 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.703536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:47.703657 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:47.703949 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:47.703954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:47.703978 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:47.704576 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:09:47.704580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:47.704596 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:09:47.704643 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:47.704987 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:47.704991 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:47.704996 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:09:47.705191 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 13:09:47.705248 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":255,"threads_started":1}
I20260809 13:09:47.706292 19142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2160ad4b5448580e3f25151ab6292" candidate_uuid: "b7c28268516944a39bd72b9d08063f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74aa52a77d254649a6e0d45cc476a36c" is_pre_election: true
I20260809 13:09:47.706650 18705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2160ad4b5448580e3f25151ab6292" candidate_uuid: "b7c28268516944a39bd72b9d08063f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f4ffca2814417eb05319d1ba05c685" is_pre_election: true
I20260809 13:09:47.706749 19142 raft_consensus.cc:2454] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c28268516944a39bd72b9d08063f79 in term 0.
I20260809 13:09:47.707064 18705 raft_consensus.cc:2454] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c28268516944a39bd72b9d08063f79 in term 0.
I20260809 13:09:47.707387 18812 leader_election.cc:302] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74aa52a77d254649a6e0d45cc476a36c, b7c28268516944a39bd72b9d08063f79; no voters:
I20260809 13:09:47.707760 19254 raft_consensus.cc:2781] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:09:47.707908 19254 raft_consensus.cc:482] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:09:47.708020 19254 raft_consensus.cc:3037] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:47.710271 19254 raft_consensus.cc:504] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.710785 19254 leader_election.cc:288] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [CANDIDATE]: Term 1 election: Requested vote from peers 74aa52a77d254649a6e0d45cc476a36c (127.10.181.131:41347), 64f4ffca2814417eb05319d1ba05c685 (127.10.181.129:41851)
I20260809 13:09:47.711176 19142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2160ad4b5448580e3f25151ab6292" candidate_uuid: "b7c28268516944a39bd72b9d08063f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74aa52a77d254649a6e0d45cc476a36c"
I20260809 13:09:47.711263 18705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37d2160ad4b5448580e3f25151ab6292" candidate_uuid: "b7c28268516944a39bd72b9d08063f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f4ffca2814417eb05319d1ba05c685"
I20260809 13:09:47.711350 19142 raft_consensus.cc:3037] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:47.711443 18705 raft_consensus.cc:3037] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:47.713541 19142 raft_consensus.cc:2454] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c28268516944a39bd72b9d08063f79 in term 1.
I20260809 13:09:47.713655 18705 raft_consensus.cc:2454] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c28268516944a39bd72b9d08063f79 in term 1.
I20260809 13:09:47.713889 18812 leader_election.cc:302] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74aa52a77d254649a6e0d45cc476a36c, b7c28268516944a39bd72b9d08063f79; no voters:
I20260809 13:09:47.714185 19254 raft_consensus.cc:2781] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:47.714581 19254 raft_consensus.cc:686] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 1 LEADER]: Becoming Leader. State: Replica: b7c28268516944a39bd72b9d08063f79, State: Running, Role: LEADER
I20260809 13:09:47.715010 19254 consensus_queue.cc:237] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } }
I20260809 13:09:47.719625 18474 catalog_manager.cc:5495] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7c28268516944a39bd72b9d08063f79 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "b7c28268516944a39bd72b9d08063f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7c28268516944a39bd72b9d08063f79" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 } health_report { overall_health: UNKNOWN } } }
W20260809 13:09:47.733666 18786 tablet.cc:2369] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:09:47.745514 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.743683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.10.181.130:45051)
0809 13:09:47.743873 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.743879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.743943 (+ 64us) server_negotiation.cc:407] Connection header received
0809 13:09:47.744106 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.744110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.744183 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.744261 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.744960 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.744964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.744967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.744982 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.745012 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.745023 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.745123 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.745185 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.745229 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:09:47.752940 18705 raft_consensus.cc:1261] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 1 FOLLOWER]: Refusing update from remote peer b7c28268516944a39bd72b9d08063f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:47.752960 19142 raft_consensus.cc:1261] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 1 FOLLOWER]: Refusing update from remote peer b7c28268516944a39bd72b9d08063f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:09:47.753801 19260 consensus_queue.cc:1035] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74aa52a77d254649a6e0d45cc476a36c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:47.754106 19254 consensus_queue.cc:1035] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f4ffca2814417eb05319d1ba05c685" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:09:47.767339 19270 mvcc.cc:205] Tried to move back new op lower bound from 7316606925827555328 to 7316606925684760576. Current Snapshot: MvccSnapshot[applied={T|T < 7316606925827555328}]
I20260809 13:09:47.768227 19268 mvcc.cc:205] Tried to move back new op lower bound from 7316606925827555328 to 7316606925684760576. Current Snapshot: MvccSnapshot[applied={T|T < 7316606925827555328}]
I20260809 13:09:47.781601 18474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:47.781836 18474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:09:47.785287 18474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0377ac9476174990ba748b8e06c6f177: Sending DeleteTablet for 3 replicas of tablet 37d2160ad4b5448580e3f25151ab6292
I20260809 13:09:47.786350 19102 tablet_service.cc:1514] Processing DeleteTablet for tablet 37d2160ad4b5448580e3f25151ab6292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:47 UTC) from {username='slave'} at 127.0.0.1:59768
I20260809 13:09:47.786468 18881 tablet_service.cc:1514] Processing DeleteTablet for tablet 37d2160ad4b5448580e3f25151ab6292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:47 UTC) from {username='slave'} at 127.0.0.1:58160
I20260809 13:09:47.787232 18669 tablet_service.cc:1514] Processing DeleteTablet for tablet 37d2160ad4b5448580e3f25151ab6292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:09:47 UTC) from {username='slave'} at 127.0.0.1:50750
I20260809 13:09:47.793116 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 18561
I20260809 13:09:47.798209 19285 tablet_replica.cc:330] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: stopping tablet replica
I20260809 13:09:47.798775 19285 raft_consensus.cc:2229] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:47.799434 19285 raft_consensus.cc:2258] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:47.801787 19285 ts_tablet_manager.cc:1905] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:47.807614 19283 tablet_replica.cc:330] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: stopping tablet replica
I20260809 13:09:47.808099 19283 raft_consensus.cc:2229] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:09:47.808830 19283 raft_consensus.cc:2258] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:47.811486 19285 ts_tablet_manager.cc:1918] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:47.812599 19284 tablet_replica.cc:330] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: stopping tablet replica
I20260809 13:09:47.812922 19283 ts_tablet_manager.cc:1905] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:47.813011 19284 raft_consensus.cc:2229] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:09:47.813704 19284 raft_consensus.cc:2258] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:09:47.815689 19284 ts_tablet_manager.cc:1905] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:09:47.819805 19283 ts_tablet_manager.cc:1918] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:47.814669 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.796331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.10.181.129:41851)
0809 13:09:47.796511 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.796518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.810879 (+ 14361us) server_negotiation.cc:407] Connection header received
0809 13:09:47.811338 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.811344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.811451 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.811534 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.813873 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.813879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.813884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.813907 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.813947 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.813964 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.814057 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.814319 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.814383 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:09:47.819979 19283 log.cc:1196] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/37d2160ad4b5448580e3f25151ab6292
I20260809 13:09:47.820376 19285 log.cc:1196] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/37d2160ad4b5448580e3f25151ab6292
I20260809 13:09:47.820638 18777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:47.821506 19283 ts_tablet_manager.cc:1939] T 37d2160ad4b5448580e3f25151ab6292 P 74aa52a77d254649a6e0d45cc476a36c: Deleting consensus metadata
I20260809 13:09:47.823400 18449 catalog_manager.cc:4841] TS 74aa52a77d254649a6e0d45cc476a36c (127.10.181.131:41347): tablet 37d2160ad4b5448580e3f25151ab6292 (table test-table [id=674b7bd05bda4d97b0aa66bc9a95d864]) successfully deleted
I20260809 13:09:47.823949 19284 ts_tablet_manager.cc:1918] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:09:47.824261 19284 log.cc:1196] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/37d2160ad4b5448580e3f25151ab6292
I20260809 13:09:47.824965 19284 ts_tablet_manager.cc:1939] T 37d2160ad4b5448580e3f25151ab6292 P b7c28268516944a39bd72b9d08063f79: Deleting consensus metadata
I20260809 13:09:47.827162 18451 catalog_manager.cc:4841] TS b7c28268516944a39bd72b9d08063f79 (127.10.181.130:45051): tablet 37d2160ad4b5448580e3f25151ab6292 (table test-table [id=674b7bd05bda4d97b0aa66bc9a95d864]) successfully deleted
I20260809 13:09:47.910624 19285 ts_tablet_manager.cc:1939] T 37d2160ad4b5448580e3f25151ab6292 P 64f4ffca2814417eb05319d1ba05c685: Deleting consensus metadata
I20260809 13:09:47.911201 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 18561
W20260809 13:09:47.939219 18449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:41851 (error 108)
I20260809 13:09:47.939536 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 18788
I20260809 13:09:47.941475 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.940909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50760)
0809 13:09:47.941127 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.941276 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 13:09:47.941761 18449 catalog_manager.cc:4584] TS 64f4ffca2814417eb05319d1ba05c685 (127.10.181.129:41851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37d2160ad4b5448580e3f25151ab6292: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
I20260809 13:09:47.942703 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.940454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.10.181.130:45051)
0809 13:09:47.940660 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:47.940666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:47.940737 (+ 71us) server_negotiation.cc:407] Connection header received
0809 13:09:47.940953 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:47.940959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:47.941031 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:47.941157 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:47.942036 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:47.942042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:47.942046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:47.942065 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:47.942104 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:47.942129 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:47.942262 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:47.942361 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 13:09:47.942426 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:09:47.943416 18979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:47.995358 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.994897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50766)
0809 13:09:47.995091 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.995197 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:09:47.996353 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:47.996020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50770)
0809 13:09:47.996160 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:47.996233 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:09:48.053906 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 18788
I20260809 13:09:48.057787 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.057301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50782)
0809 13:09:48.057530 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:48.057633 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 13:09:48.058868 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.058532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50784)
0809 13:09:48.058679 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:48.058753 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 13:09:48.080332 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 18994
I20260809 13:09:48.083191 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.081174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.10.181.131:41347)
0809 13:09:48.081354 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:48.081359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:48.081392 (+ 33us) server_negotiation.cc:407] Connection header received
0809 13:09:48.081628 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:48.081632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:48.081713 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:48.081793 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:48.082594 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:48.082598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:48.082601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:48.082616 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:48.082657 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:48.082674 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:48.082778 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:48.082876 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 13:09:48.082924 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:09:48.083886 19205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:48.139312 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.138743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50798)
0809 13:09:48.138983 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:48.139113 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 13:09:48.140465 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.139993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41851 (local address 127.0.0.1:50800)
0809 13:09:48.140239 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:48.140324 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 13:09:48.171579 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 18994
I20260809 13:09:48.198186 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 18422
I20260809 13:09:48.199270 18518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:48.304281 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 18422
2026-08-09T13:09:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2544 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 13:09:48.352061 10966 mini_oidc.cc:152] Starting JWKS server
I20260809 13:09:48.354923 10966 webserver.cc:460] Webserver started at https://127.0.0.1:45247/ using document root <none> and password file <none>
I20260809 13:09:48.355264 10966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 13:09:48.355855 10966 webserver.cc:460] Webserver started at http://127.0.0.1:42569/ using document root <none> and password file <none>
2026-08-09T13:09:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:48Z Disabled control of system clock
I20260809 13:09:48.387042 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:38717
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:44497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:38717
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/testchainca.cert with env {}
W20260809 13:09:48.704208 19300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:48.704468 19300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:48.704559 19300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:48.711495 19300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:48.711588 19300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:48.711640 19300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:48.711695 19300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:48.711740 19300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 13:09:48.711798 19300 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 13:09:48.711836 19300 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45247/jwks.json
I20260809 13:09:48.721338 19300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:38717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:48.722503 19300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:48.723804 19300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:48.734998 19312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:48.735553 19313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:48.735605 19317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:48.736307 19300 server_base.cc:1034] running on GCE node
I20260809 13:09:48.737063 19300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:48.738430 19300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:48.739630 19300 hybrid_clock.cc:648] HybridClock initialized: now 1786280988739581 us; error 39 us; skew 500 ppm
I20260809 13:09:48.739959 19300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:48.742323 19300 webserver.cc:460] Webserver started at http://127.10.181.190:37939/ using document root <none> and password file <none>
I20260809 13:09:48.742988 19300 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:48.743099 19300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:48.743403 19300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:48.746260 19300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "cc113b14c4e64b589d925b49b5091b67"
format_stamp: "Formatted at 2026-08-09 13:09:48 on dist-test-slave-42z9"
server_key: "3d1830559979fe6fe9155f35f2907e43"
server_key_iv: "2ba783c21678f62426d5d1e8c4f53789"
server_key_version: "encryptionkey@0"
I20260809 13:09:48.747014 19300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "cc113b14c4e64b589d925b49b5091b67"
format_stamp: "Formatted at 2026-08-09 13:09:48 on dist-test-slave-42z9"
server_key: "3d1830559979fe6fe9155f35f2907e43"
server_key_iv: "2ba783c21678f62426d5d1e8c4f53789"
server_key_version: "encryptionkey@0"
I20260809 13:09:48.752087 19300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:09:48.755438 19324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:48.756670 19300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:09:48.756841 19300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "cc113b14c4e64b589d925b49b5091b67"
format_stamp: "Formatted at 2026-08-09 13:09:48 on dist-test-slave-42z9"
server_key: "3d1830559979fe6fe9155f35f2907e43"
server_key_iv: "2ba783c21678f62426d5d1e8c4f53789"
server_key_version: "encryptionkey@0"
I20260809 13:09:48.757030 19300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:48.783411 19300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:48.784169 19300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:48.784456 19300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:48.801321 19300 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:38717
I20260809 13:09:48.801331 19405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:38717 every 8 connection(s)
I20260809 13:09:48.802577 19300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:48.803951 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 19300
I20260809 13:09:48.804141 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:48.804771 10966 external_mini_cluster.cc:1428] Setting key 17321a7fb353d445c33f751fd8ba5469
I20260809 13:09:48.806582 19406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:48.817881 19409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.806531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.10.181.190:38717)
0809 13:09:48.807487 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:48.807498 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:48.807540 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:09:48.809111 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:48.809141 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:48.809629 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:48.809944 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:48.810534 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.811830 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.813743 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.814368 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.814493 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:48.815706 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:48.815713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:48.815729 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:48.816267 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:48.816345 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:48.816366 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:48.816459 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:48.816682 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 13:09:48.816977 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":348,"threads_started":1}
I20260809 13:09:48.818895 19406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67: Bootstrap starting.
I20260809 13:09:48.821367 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:09:48.822108 19406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:48.823256 19406 log.cc:824] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:48.826006 19406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67: No bootstrap required, opened a new log
I20260809 13:09:48.832973 19406 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } }
I20260809 13:09:48.833294 19406 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:48.833366 19406 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc113b14c4e64b589d925b49b5091b67, State: Initialized, Role: FOLLOWER
I20260809 13:09:48.833869 19406 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } }
I20260809 13:09:48.834077 19406 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:48.834172 19406 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:48.834297 19406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:48.836249 19406 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } }
I20260809 13:09:48.836678 19406 leader_election.cc:302] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc113b14c4e64b589d925b49b5091b67; no voters:
I20260809 13:09:48.837250 19406 leader_election.cc:288] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:48.837383 19414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:48.838106 19414 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [term 1 LEADER]: Becoming Leader. State: Replica: cc113b14c4e64b589d925b49b5091b67, State: Running, Role: LEADER
I20260809 13:09:48.838594 19414 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } }
I20260809 13:09:48.839244 19406 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:48.841372 19416 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc113b14c4e64b589d925b49b5091b67. Latest consensus state: current_term: 1 leader_uuid: "cc113b14c4e64b589d925b49b5091b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } } }
I20260809 13:09:48.841830 19416 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:48.841652 19415 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc113b14c4e64b589d925b49b5091b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc113b14c4e64b589d925b49b5091b67" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 38717 } } }
I20260809 13:09:48.841990 19415 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:48.843626 19426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:48.848776 19426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:48.856817 19426 catalog_manager.cc:1347] Generated new cluster ID: b702120d88f54747b1d2bd82173a1e00
I20260809 13:09:48.856909 19426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:48.877810 19426 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:48.879416 19426 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:48.887997 19426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc113b14c4e64b589d925b49b5091b67: Generated new TSK 0
I20260809 13:09:48.888705 19426 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 13:09:48.923918 19456 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 13:09:48.924139 19456 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 13:09:48.925006 19409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:48.904081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.10.181.190:38717)
0809 13:09:48.904274 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:48.904281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:48.904837 (+ 556us) server_negotiation.cc:407] Connection header received
0809 13:09:48.905240 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:48.905244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:48.905309 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:48.905414 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0809 13:09:48.906091 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.906801 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.908861 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.909299 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.909385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:48.910007 (+ 622us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 13:09:48.924523 (+ 14516us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 13:09:48.924783 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 13:09:48.925118 19409 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 13:09:48.927073 19455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:48.904080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:38717 (local address 127.0.0.1:54678)
0809 13:09:48.904733 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:48.904753 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:48.904959 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:48.905496 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:48.905502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:48.905516 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 13:09:48.905950 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:48.905963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.906930 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.906936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:48.908726 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:48.908742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:48.909486 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:48.909494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:48.909678 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:48.909755 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 13:09:48.924788 (+ 15033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 13:09:48.925121 (+ 333us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.181.190:38717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":311,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.181.190:38717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 13:09:48.928519 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 19300
I20260809 13:09:48.929766 19400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:49.021055 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 19300
2026-08-09T13:09:49Z chronyd exiting
I20260809 13:09:49.079746 10966 test_util.cc:175] -----------------------------------------------
I20260809 13:09:49.079878 10966 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786280946828509-10966-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 13:09:49.083660 10966 mini_oidc.cc:152] Starting JWKS server
I20260809 13:09:49.085377 10966 webserver.cc:460] Webserver started at https://127.0.0.1:46421/ using document root <none> and password file <none>
I20260809 13:09:49.085609 10966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 13:09:49.086229 10966 webserver.cc:460] Webserver started at http://127.0.0.1:40667/ using document root <none> and password file <none>
2026-08-09T13:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:49Z Disabled control of system clock
I20260809 13:09:49.119467 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:35505
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:36795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:35505
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46421/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/testchainca.cert with env {}
W20260809 13:09:49.438922 19467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:49.439178 19467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:49.439270 19467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:49.446285 19467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:49.446381 19467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:49.446425 19467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:49.446473 19467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:49.446516 19467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 13:09:49.446574 19467 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 13:09:49.446614 19467 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46421/jwks.json
I20260809 13:09:49.456049 19467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:35505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:35505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46421/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:49.457206 19467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:49.458433 19467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:49.469339 19477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:49.469482 19475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:49.469460 19480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:49.470197 19467 server_base.cc:1034] running on GCE node
I20260809 13:09:49.470865 19467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:49.472074 19467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:49.473251 19467 hybrid_clock.cc:648] HybridClock initialized: now 1786280989473203 us; error 39 us; skew 500 ppm
I20260809 13:09:49.473600 19467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:49.479539 19467 webserver.cc:460] Webserver started at http://127.10.181.190:42011/ using document root <none> and password file <none>
I20260809 13:09:49.480159 19467 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:49.480265 19467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:49.480583 19467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:49.483266 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "7a72b680baaf4f9b86ef2a7927270136"
format_stamp: "Formatted at 2026-08-09 13:09:49 on dist-test-slave-42z9"
server_key: "8a72abc658a05bce8332ccaa15f49d47"
server_key_iv: "5e93e9ba4e40518416accf18a1a51544"
server_key_version: "encryptionkey@0"
I20260809 13:09:49.484009 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "7a72b680baaf4f9b86ef2a7927270136"
format_stamp: "Formatted at 2026-08-09 13:09:49 on dist-test-slave-42z9"
server_key: "8a72abc658a05bce8332ccaa15f49d47"
server_key_iv: "5e93e9ba4e40518416accf18a1a51544"
server_key_version: "encryptionkey@0"
I20260809 13:09:49.488911 19467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:09:49.492138 19489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:49.493399 19467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:49.493613 19467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "7a72b680baaf4f9b86ef2a7927270136"
format_stamp: "Formatted at 2026-08-09 13:09:49 on dist-test-slave-42z9"
server_key: "8a72abc658a05bce8332ccaa15f49d47"
server_key_iv: "5e93e9ba4e40518416accf18a1a51544"
server_key_version: "encryptionkey@0"
I20260809 13:09:49.493818 19467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:49.508574 19467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:49.509275 19467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:49.509584 19467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:49.526055 19467 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:35505
I20260809 13:09:49.526095 19571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:35505 every 8 connection(s)
I20260809 13:09:49.527256 19467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:49.530772 19572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:49.537545 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 19467
I20260809 13:09:49.537717 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:49.538349 10966 external_mini_cluster.cc:1428] Setting key a05881ec728a71e4a918e6803fdeb76d
I20260809 13:09:49.540637 19572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136: Bootstrap starting.
I20260809 13:09:49.543772 19572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:49.544884 19572 log.cc:824] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:49.550217 19572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136: No bootstrap required, opened a new log
I20260809 13:09:49.551277 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:49.540008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.10.181.190:35505)
0809 13:09:49.540937 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:49.540947 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:49.540987 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:09:49.542741 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:49.542764 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:49.543162 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:49.543451 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:49.544039 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.545344 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.546776 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.547395 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.547572 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:49.548637 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:49.548647 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:49.548664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:49.549297 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:49.549396 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:49.549491 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:49.549618 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:49.549789 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 13:09:49.550090 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":314,"threads_started":1}
I20260809 13:09:49.554804 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:09:49.558665 19572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } }
I20260809 13:09:49.558964 19572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:49.559032 19572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a72b680baaf4f9b86ef2a7927270136, State: Initialized, Role: FOLLOWER
I20260809 13:09:49.559543 19572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } }
I20260809 13:09:49.559747 19572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:49.559847 19572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:49.559979 19572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:49.562217 19572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } }
I20260809 13:09:49.562631 19572 leader_election.cc:302] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a72b680baaf4f9b86ef2a7927270136; no voters:
I20260809 13:09:49.563092 19572 leader_election.cc:288] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:49.563216 19578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:49.563879 19578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [term 1 LEADER]: Becoming Leader. State: Replica: 7a72b680baaf4f9b86ef2a7927270136, State: Running, Role: LEADER
I20260809 13:09:49.564340 19578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } }
I20260809 13:09:49.564846 19572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:49.567102 19580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a72b680baaf4f9b86ef2a7927270136. Latest consensus state: current_term: 1 leader_uuid: "7a72b680baaf4f9b86ef2a7927270136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } } }
I20260809 13:09:49.567459 19580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:49.567750 19579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a72b680baaf4f9b86ef2a7927270136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72b680baaf4f9b86ef2a7927270136" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 35505 } } }
I20260809 13:09:49.568074 19579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:49.570681 19588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:49.575538 19588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:49.583189 19588 catalog_manager.cc:1347] Generated new cluster ID: a950917bc613423181e6e9d92bd9d15d
I20260809 13:09:49.583285 19588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:49.613804 19588 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:49.615015 19588 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:49.623729 19588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a72b680baaf4f9b86ef2a7927270136: Generated new TSK 0
I20260809 13:09:49.624596 19588 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 13:09:49.668193 19613 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 13:09:49.668323 19613 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 13:09:49.669215 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:49.643271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.10.181.190:35505)
0809 13:09:49.643417 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:49.643423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:49.643887 (+ 464us) server_negotiation.cc:407] Connection header received
0809 13:09:49.644111 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:49.644115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:49.644194 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:49.644318 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0809 13:09:49.644771 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.645507 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.646837 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.647292 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.647403 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:49.647868 (+ 465us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 13:09:49.668604 (+ 20736us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46421/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 13:09:49.668930 (+ 326us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51028: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46421/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 13:09:49.669353 19576 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51028: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46421/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 13:09:49.669396 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:49.643222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:35505 (local address 127.0.0.1:51028)
0809 13:09:49.643788 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:49.643806 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:49.643940 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:49.644351 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:49.644355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:49.644367 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0809 13:09:49.644612 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:49.644621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.645625 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.645630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:49.646694 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:49.646705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:49.647446 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:49.647453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:49.647609 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:49.647642 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 13:09:49.668797 (+ 21155us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46421/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 13:09:49.669025 (+ 228us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.181.190:35505: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46421/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":317,"threads_started":1}
I20260809 13:09:49.670935 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 19467
I20260809 13:09:49.671943 19566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:49.769347 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 19467
2026-08-09T13:09:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (748 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 13:09:49.831977 10966 mini_oidc.cc:152] Starting JWKS server
I20260809 13:09:49.833757 10966 webserver.cc:460] Webserver started at https://127.0.0.1:45229/ using document root <none> and password file <none>
I20260809 13:09:49.833961 10966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 13:09:49.834535 10966 webserver.cc:460] Webserver started at http://127.0.0.1:35701/ using document root <none> and password file <none>
2026-08-09T13:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:49Z Disabled control of system clock
I20260809 13:09:49.866496 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37251
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:37757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:37251
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45229/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 13:09:50.193890 19623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:50.194181 19623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:50.194324 19623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:50.204072 19623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:50.204193 19623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:50.204258 19623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:50.204339 19623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:50.204406 19623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 13:09:50.204491 19623 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 13:09:50.204559 19623 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45229/jwks.json
I20260809 13:09:50.218541 19623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:37251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45229/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:50.220274 19623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:50.222218 19623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:50.233971 19635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:50.234007 19637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:50.234143 19639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:50.234670 19623 server_base.cc:1034] running on GCE node
I20260809 13:09:50.235443 19623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:50.236845 19623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:50.238044 19623 hybrid_clock.cc:648] HybridClock initialized: now 1786280990237994 us; error 42 us; skew 500 ppm
I20260809 13:09:50.238365 19623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:50.240490 19623 webserver.cc:460] Webserver started at http://127.10.181.190:45523/ using document root <none> and password file <none>
I20260809 13:09:50.241146 19623 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:50.241235 19623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:50.241544 19623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:50.244602 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "d3559689cae6475da207c0c8eebcc1f9"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "52afec9ea79e7b94eac33774e3d1d7c7"
server_key_iv: "30124134e0041a4f77d4f5c296099424"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.245352 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "d3559689cae6475da207c0c8eebcc1f9"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "52afec9ea79e7b94eac33774e3d1d7c7"
server_key_iv: "30124134e0041a4f77d4f5c296099424"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.250356 19623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 13:09:50.253629 19645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:50.254895 19623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:09:50.255062 19623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "d3559689cae6475da207c0c8eebcc1f9"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "52afec9ea79e7b94eac33774e3d1d7c7"
server_key_iv: "30124134e0041a4f77d4f5c296099424"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.255259 19623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:50.280630 19623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:50.281535 19623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:50.281853 19623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:50.299576 19623 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:37251
I20260809 13:09:50.299602 19728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:37251 every 8 connection(s)
I20260809 13:09:50.300899 19623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:50.304641 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 19623
I20260809 13:09:50.304719 19729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:50.304843 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:50.305500 10966 external_mini_cluster.cc:1428] Setting key 7885c6b48db451bec0e91d5ec9fbfded
I20260809 13:09:50.317899 19729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9: Bootstrap starting.
I20260809 13:09:50.319270 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:50.307541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.10.181.190:37251)
0809 13:09:50.308522 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:50.308532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:50.308577 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:09:50.310471 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:50.310496 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:50.310955 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:50.311285 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:50.311930 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.313166 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.315008 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.315710 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.315838 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:50.317019 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:50.317027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:50.317041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:50.317585 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:50.317677 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:50.317701 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:50.317787 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:50.318028 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 13:09:50.318348 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":363,"threads_started":1}
I20260809 13:09:50.321307 19729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:50.322495 19729 log.cc:824] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:50.323232 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:09:50.325314 19729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9: No bootstrap required, opened a new log
I20260809 13:09:50.332933 19729 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } }
I20260809 13:09:50.333263 19729 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:50.333341 19729 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3559689cae6475da207c0c8eebcc1f9, State: Initialized, Role: FOLLOWER
I20260809 13:09:50.333853 19729 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } }
I20260809 13:09:50.334069 19729 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:50.334170 19729 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:50.334287 19729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:50.336673 19729 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } }
I20260809 13:09:50.337113 19729 leader_election.cc:302] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3559689cae6475da207c0c8eebcc1f9; no voters:
I20260809 13:09:50.337672 19729 leader_election.cc:288] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:50.337783 19737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:50.338478 19737 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [term 1 LEADER]: Becoming Leader. State: Replica: d3559689cae6475da207c0c8eebcc1f9, State: Running, Role: LEADER
I20260809 13:09:50.339015 19737 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } }
I20260809 13:09:50.339583 19729 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:50.341682 19740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3559689cae6475da207c0c8eebcc1f9. Latest consensus state: current_term: 1 leader_uuid: "d3559689cae6475da207c0c8eebcc1f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } } }
I20260809 13:09:50.342216 19740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:50.343667 19738 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3559689cae6475da207c0c8eebcc1f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3559689cae6475da207c0c8eebcc1f9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37251 } } }
I20260809 13:09:50.343999 19738 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:50.345207 19754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:50.349599 19754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:50.359513 19754 catalog_manager.cc:1347] Generated new cluster ID: 646cbdb417e14fc3aec6a00a54cd6fe0
I20260809 13:09:50.359627 19754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:50.376716 19754 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:50.378542 19754 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:50.397277 19754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3559689cae6475da207c0c8eebcc1f9: Generated new TSK 0
I20260809 13:09:50.398379 19754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:09:50.420435 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:50.412096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.10.181.190:37251)
0809 13:09:50.412286 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:50.412293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:50.412660 (+ 367us) server_negotiation.cc:407] Connection header received
0809 13:09:50.412856 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:50.412861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:50.412963 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:50.413131 (+ 168us) server_negotiation.cc:226] Negotiated authn=JWT
0809 13:09:50.413630 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.414764 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.416083 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.416760 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.416876 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:50.417396 (+ 520us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 13:09:50.419768 (+ 2372us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45229/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 13:09:50.420112 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48504: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45229/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 13:09:50.420560 19735 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48504: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45229/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 13:09:50.422183 19777 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:50.411960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:37251 (local address 127.0.0.1:48504)
0809 13:09:50.412566 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:50.412581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:50.412697 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:50.413157 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:50.413162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:50.413172 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 13:09:50.413483 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:50.413496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.414919 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.414923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:50.415942 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:50.415955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:50.416946 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:50.416952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:50.417096 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:50.417129 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 13:09:50.419969 (+ 2840us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45229/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 13:09:50.421768 (+ 1799us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.181.190:37251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45229/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":1529,"thread_start_us":341,"threads_started":1}
I20260809 13:09:50.423629 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 19623
I20260809 13:09:50.424844 19721 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 13:09:50.431599 19778 webserver.cc:533] Webserver: sslize failed:
E20260809 13:09:50.431682 19778 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 13:09:50.517376 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 19623
2026-08-09T13:09:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (748 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T13:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:50Z Disabled control of system clock
I20260809 13:09:50.613265 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36705
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:36705
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/testchainca.cert with env {}
W20260809 13:09:50.936721 19785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:50.937021 19785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:50.937116 19785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:50.944242 19785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:50.944342 19785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:50.944387 19785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:50.944468 19785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:50.944542 19785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 13:09:50.944626 19785 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 13:09:50.944684 19785 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 13:09:50.954357 19785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:36705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:50.955641 19785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:50.956885 19785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:50.968297 19794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:50.968380 19793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:50.968340 19796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:50.969507 19785 server_base.cc:1034] running on GCE node
I20260809 13:09:50.970281 19785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:50.971613 19785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:50.972829 19785 hybrid_clock.cc:648] HybridClock initialized: now 1786280990972775 us; error 37 us; skew 500 ppm
I20260809 13:09:50.973160 19785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:09:50.976646 19785 webserver.cc:460] Webserver started at http://127.10.181.190:36747/ using document root <none> and password file <none>
I20260809 13:09:50.977613 19785 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:50.977735 19785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:50.978109 19785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:50.982350 19785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "68b1ac62c45843adaf1793d035c806c4"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "078b3b4b35770ed39cec05424873c339"
server_key_iv: "26fa120ca88e747350041e6c3486b735"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.983448 19785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "68b1ac62c45843adaf1793d035c806c4"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "078b3b4b35770ed39cec05424873c339"
server_key_iv: "26fa120ca88e747350041e6c3486b735"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.991668 19785 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260809 13:09:50.996415 19805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:50.997870 19785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:09:50.998041 19785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "68b1ac62c45843adaf1793d035c806c4"
format_stamp: "Formatted at 2026-08-09 13:09:50 on dist-test-slave-42z9"
server_key: "078b3b4b35770ed39cec05424873c339"
server_key_iv: "26fa120ca88e747350041e6c3486b735"
server_key_version: "encryptionkey@0"
I20260809 13:09:50.998263 19785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:51.027133 19785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:51.028016 19785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:51.028295 19785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:51.046669 19785 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:36705
I20260809 13:09:51.046698 19890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:36705 every 8 connection(s)
I20260809 13:09:51.048894 19785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:51.053134 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 19785
I20260809 13:09:51.053347 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:51.053994 19891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:51.054227 10966 external_mini_cluster.cc:1428] Setting key 2da111611f5d24f9b6c62f686259e913
I20260809 13:09:51.069199 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:51.055970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.10.181.190:36705)
0809 13:09:51.057009 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:51.057024 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:51.057075 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:09:51.059225 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:51.059254 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:51.059889 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:51.060336 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:51.060874 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.062900 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.064168 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.065147 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.065322 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:51.066402 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:51.066416 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:51.066433 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:09:51.067167 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:51.067261 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:09:51.067286 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:51.067421 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:51.067631 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 13:09:51.067968 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":317,"threads_started":1}
I20260809 13:09:51.071178 19891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4: Bootstrap starting.
I20260809 13:09:51.073400 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:09:51.074734 19891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:51.075819 19891 log.cc:824] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:51.078590 19891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4: No bootstrap required, opened a new log
I20260809 13:09:51.086282 19891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } }
I20260809 13:09:51.086609 19891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:51.086679 19891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b1ac62c45843adaf1793d035c806c4, State: Initialized, Role: FOLLOWER
I20260809 13:09:51.087143 19891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } }
I20260809 13:09:51.087340 19891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:51.087435 19891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:51.087556 19891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:51.090039 19891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } }
I20260809 13:09:51.090561 19891 leader_election.cc:302] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b1ac62c45843adaf1793d035c806c4; no voters:
I20260809 13:09:51.091068 19891 leader_election.cc:288] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:51.091205 19901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:51.091877 19901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [term 1 LEADER]: Becoming Leader. State: Replica: 68b1ac62c45843adaf1793d035c806c4, State: Running, Role: LEADER
I20260809 13:09:51.092411 19901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } }
I20260809 13:09:51.092932 19891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:51.095217 19902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b1ac62c45843adaf1793d035c806c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } } }
I20260809 13:09:51.095297 19903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b1ac62c45843adaf1793d035c806c4. Latest consensus state: current_term: 1 leader_uuid: "68b1ac62c45843adaf1793d035c806c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b1ac62c45843adaf1793d035c806c4" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36705 } } }
I20260809 13:09:51.095602 19902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:51.095660 19903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:51.098097 19915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:51.103601 19915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:51.113022 19915 catalog_manager.cc:1347] Generated new cluster ID: 7e56f33b58834c0fb93ea10876885a09
I20260809 13:09:51.113139 19915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:51.137580 19915 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:51.138803 19915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:51.147992 19915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b1ac62c45843adaf1793d035c806c4: Generated new TSK 0
I20260809 13:09:51.148963 19915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:09:51.227545 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:51.172895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.10.181.190:36705)
0809 13:09:51.173093 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:51.173099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:51.173996 (+ 897us) server_negotiation.cc:407] Connection header received
0809 13:09:51.174211 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:51.174217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:51.174317 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:51.174480 (+ 163us) server_negotiation.cc:226] Negotiated authn=JWT
0809 13:09:51.174874 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.175988 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.177318 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.178020 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.178152 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:51.178616 (+ 464us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 13:09:51.226765 (+ 48149us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 13:09:51.227212 (+ 447us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39992: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 13:09:51.227695 19898 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39992: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 13:09:51.227679 19943 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:51.172776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36705 (local address 127.0.0.1:39992)
0809 13:09:51.173901 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:51.173917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:51.174033 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:51.174477 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:51.174480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:51.174489 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 13:09:51.174735 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:51.174742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.176116 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.176119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:51.177181 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:51.177193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:51.178170 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:51.178175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:51.178291 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:51.178324 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 13:09:51.227061 (+ 48737us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 13:09:51.227321 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.181.190:36705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":1003,"thread_start_us":849,"threads_started":1}
I20260809 13:09:51.229125 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 19785
I20260809 13:09:51.230286 19884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:51.328194 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 19785
2026-08-09T13:09:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (790 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (417 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (416 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:52 dist-test-slave-42z9 krb5kdc[19966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:52 dist-test-slave-42z9 krb5kdc[19966](info): set up 2 sockets
Aug 09 13:09:52 dist-test-slave-42z9 krb5kdc[19966](info): commencing operation
krb5kdc: starting...
W20260809 13:09:53.372884 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:53 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:09:53.600664 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:44431
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:44431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:53.923120 19983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:53.923399 19983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:53.923496 19983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:53.930508 19983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:53.930605 19983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:53.930658 19983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:53.930713 19983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:53.930756 19983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:53.940208 19983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:44431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:44431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:53.941428 19983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:53.942651 19983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:53.953960 19994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:53.953994 19996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:53.954053 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:53.954851 19983 server_base.cc:1034] running on GCE node
I20260809 13:09:53.955549 19983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:53.956779 19983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:53.957978 19983 hybrid_clock.cc:648] HybridClock initialized: now 1786280993957936 us; error 29 us; skew 500 ppm
I20260809 13:09:53.958302 19983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:53 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280993, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:53.961238 19983 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:09:53.962688 19983 webserver.cc:460] Webserver started at http://127.10.181.190:43045/ using document root <none> and password file <none>
I20260809 13:09:53.963318 19983 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:53.963442 19983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:53.963745 19983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:53.966624 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "57daeca8cf0e422793a0a2d15734e736"
format_stamp: "Formatted at 2026-08-09 13:09:53 on dist-test-slave-42z9"
server_key: "2dadfa960dc8606a5ac156d57a9d0437"
server_key_iv: "a755acf19f0671ad0cc4bb26d5096ed1"
server_key_version: "encryptionkey@0"
I20260809 13:09:53.967361 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "57daeca8cf0e422793a0a2d15734e736"
format_stamp: "Formatted at 2026-08-09 13:09:53 on dist-test-slave-42z9"
server_key: "2dadfa960dc8606a5ac156d57a9d0437"
server_key_iv: "a755acf19f0671ad0cc4bb26d5096ed1"
server_key_version: "encryptionkey@0"
I20260809 13:09:53.972666 19983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:09:53.975910 20008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:53.977111 19983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 13:09:53.977272 19983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "57daeca8cf0e422793a0a2d15734e736"
format_stamp: "Formatted at 2026-08-09 13:09:53 on dist-test-slave-42z9"
server_key: "2dadfa960dc8606a5ac156d57a9d0437"
server_key_iv: "a755acf19f0671ad0cc4bb26d5096ed1"
server_key_version: "encryptionkey@0"
I20260809 13:09:53.977478 19983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:54.008293 19983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:54.012053 19983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:54.012316 19983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:54.029153 19983 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:44431
I20260809 13:09:54.029191 20081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:44431 every 8 connection(s)
I20260809 13:09:54.030468 19983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:54.034133 20083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:09:54.040423 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 19983
I20260809 13:09:54.040611 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:54.041219 10966 external_mini_cluster.cc:1428] Setting key 0787d0bc27e24a4070eb7cff50b72e1d
I20260809 13:09:54.044477 20083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736: Bootstrap starting.
I20260809 13:09:54.047937 20083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:54.049022 20083 log.cc:824] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:54.051666 20083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736: No bootstrap required, opened a new log
Aug 09 13:09:54 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:54.060289 20083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } }
I20260809 13:09:54.060690 20083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:54.060811 20083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57daeca8cf0e422793a0a2d15734e736, State: Initialized, Role: FOLLOWER
I20260809 13:09:54.061532 20083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } }
I20260809 13:09:54.061826 20083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:54.061967 20083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:54.062165 20083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:54.063390 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:54.043134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.10.181.190:44431)
0809 13:09:54.043855 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:54.043866 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:54.043912 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:09:54.045662 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:54.045683 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:54.046049 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:54.046366 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:54.048079 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.049394 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.050637 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.051359 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.051528 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:54.055822 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:54.055830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:54.055848 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:54.055882 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:54.058877 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:54.059681 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:54.059687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:54.059693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:54.059848 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:54.060229 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:54.060232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:54.060234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:54.060791 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:54.061082 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:54.061594 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 13:09:54.062049 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":325,"threads_started":1}
I20260809 13:09:54.064865 20083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } }
I20260809 13:09:54.065343 20083 leader_election.cc:302] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57daeca8cf0e422793a0a2d15734e736; no voters:
I20260809 13:09:54.065995 20083 leader_election.cc:288] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:54.066121 20093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:54.066843 20093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [term 1 LEADER]: Becoming Leader. State: Replica: 57daeca8cf0e422793a0a2d15734e736, State: Running, Role: LEADER
I20260809 13:09:54.067394 20093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } }
I20260809 13:09:54.068154 20083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:54.070431 20097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57daeca8cf0e422793a0a2d15734e736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } } }
I20260809 13:09:54.070398 20098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57daeca8cf0e422793a0a2d15734e736. Latest consensus state: current_term: 1 leader_uuid: "57daeca8cf0e422793a0a2d15734e736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57daeca8cf0e422793a0a2d15734e736" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44431 } } }
I20260809 13:09:54.070818 20098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:54.070815 20097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:54.072326 20105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:54.077105 20105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:54.086381 20105 catalog_manager.cc:1347] Generated new cluster ID: 266153f93c6e4b889bf56a2a0e68a6c0
I20260809 13:09:54.086488 20105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:54.112807 20105 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:54.114679 20105 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:54.137207 20105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57daeca8cf0e422793a0a2d15734e736: Generated new TSK 0
I20260809 13:09:54.138087 20105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:09:54.206887 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:54.535238 20127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:54.535502 20127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:54.535640 20127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:54.542743 20127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:54.542846 20127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:54.542999 20127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:54.552379 20127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:54.553586 20127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:54.554862 20127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:54.567312 20133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:54.567349 20135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:54.567366 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:54.567950 20127 server_base.cc:1034] running on GCE node
I20260809 13:09:54.568562 20127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:54.569515 20127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:54.570713 20127 hybrid_clock.cc:648] HybridClock initialized: now 1786280994570674 us; error 38 us; skew 500 ppm
I20260809 13:09:54.571008 20127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:54 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280994, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:54.578251 20127 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:09:54.579634 20127 webserver.cc:460] Webserver started at http://127.10.181.129:42405/ using document root <none> and password file <none>
I20260809 13:09:54.580296 20127 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:54.580418 20127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:54.580718 20127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:54.583796 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "32092fadcc084626b28cba7b3a85f35a"
format_stamp: "Formatted at 2026-08-09 13:09:54 on dist-test-slave-42z9"
server_key: "640d2c7fea9c650d771015bd7df6cb69"
server_key_iv: "926869fee3b1a196c16857823b26a24a"
server_key_version: "encryptionkey@0"
I20260809 13:09:54.584553 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "32092fadcc084626b28cba7b3a85f35a"
format_stamp: "Formatted at 2026-08-09 13:09:54 on dist-test-slave-42z9"
server_key: "640d2c7fea9c650d771015bd7df6cb69"
server_key_iv: "926869fee3b1a196c16857823b26a24a"
server_key_version: "encryptionkey@0"
I20260809 13:09:54.589476 20127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:09:54.592706 20149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:54.593935 20127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 13:09:54.594102 20127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "32092fadcc084626b28cba7b3a85f35a"
format_stamp: "Formatted at 2026-08-09 13:09:54 on dist-test-slave-42z9"
server_key: "640d2c7fea9c650d771015bd7df6cb69"
server_key_iv: "926869fee3b1a196c16857823b26a24a"
server_key_version: "encryptionkey@0"
I20260809 13:09:54.594278 20127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:54.615370 20127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:54.619082 20127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:54.619347 20127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:54.620265 20127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:54.621534 20127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:54.621613 20127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:54.621733 20127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:54.621788 20127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:54.653821 20127 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:36857
I20260809 13:09:54.653883 20326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:36857 every 8 connection(s)
I20260809 13:09:54.655117 20127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:54.659910 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 20127
I20260809 13:09:54.660081 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:54.660691 10966 external_mini_cluster.cc:1428] Setting key 4e270655c0b64f275d3a3f9757dce143
Aug 09 13:09:54 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280994, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:54.673976 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:54.657509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:39147 (local address 127.10.181.190:44431)
0809 13:09:54.657925 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:54.657929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:54.658794 (+ 865us) server_negotiation.cc:407] Connection header received
0809 13:09:54.660677 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:54.660683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:54.660768 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:54.660876 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:54.662611 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.663317 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.664785 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.665167 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.665278 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:54.668568 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:54.668580 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:54.668583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:54.668615 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:54.671065 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:54.671739 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:54.671743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:54.671745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:54.671861 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:54.672285 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:54.672289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:54.672291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:54.672546 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:54.672685 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:54.673360 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 13:09:54.673626 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":207,"threads_started":1}
I20260809 13:09:54.674906 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:54.657950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44431 (local address 127.10.181.129:39147)
0809 13:09:54.658646 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:54.658683 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:54.660414 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:54.661113 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:54.661121 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:54.661677 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:54.662438 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:54.662463 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.663471 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.663475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:54.664649 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:54.664661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:54.665385 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:54.665393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:54.665648 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:54.666241 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:54.666264 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:54.668338 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:54.671196 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:54.671205 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:54.671220 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:54.671616 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:54.671964 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:54.671970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:54.671974 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:54.672173 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:54.672677 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:54.672684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:54.672984 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 13:09:54.673218 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 13:09:54.673725 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":265,"threads_started":1}
I20260809 13:09:54.676499 20330 heartbeater.cc:347] Connected to a master server at 127.10.181.190:44431
I20260809 13:09:54.676846 20330 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:54.677754 20330 heartbeater.cc:511] Master 127.10.181.190:44431 requested a full tablet report, sending...
I20260809 13:09:54.680298 20029 ts_manager.cc:194] Registered new tserver with Master: 32092fadcc084626b28cba7b3a85f35a (127.10.181.129:36857)
I20260809 13:09:54.682391 20029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:39147
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:09:54.715947 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:55.038300 20337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:55.038573 20337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:55.038702 20337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:55.045693 20337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:55.045800 20337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:55.045951 20337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:55.055220 20337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:55.056384 20337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:55.057634 20337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:55.070447 20349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:55.070473 20348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:55.071153 20352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:55.071127 20337 server_base.cc:1034] running on GCE node
I20260809 13:09:55.071808 20337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:55.072727 20337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:55.073917 20337 hybrid_clock.cc:648] HybridClock initialized: now 1786280995073866 us; error 45 us; skew 500 ppm
I20260809 13:09:55.074231 20337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.077332 20337 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:09:55.078687 20337 webserver.cc:460] Webserver started at http://127.10.181.130:46273/ using document root <none> and password file <none>
I20260809 13:09:55.079317 20337 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:55.079432 20337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:55.079699 20337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:55.082671 20337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "bed508425fef405fabffcddd28aed01a"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "6d851319d7cbbce176b7b64128ba6dd0"
server_key_iv: "7007f758500328424369644e5e86b594"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.083369 20337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "bed508425fef405fabffcddd28aed01a"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "6d851319d7cbbce176b7b64128ba6dd0"
server_key_iv: "7007f758500328424369644e5e86b594"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.088213 20337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:09:55.091538 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.092835 20337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:09:55.092983 20337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "bed508425fef405fabffcddd28aed01a"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "6d851319d7cbbce176b7b64128ba6dd0"
server_key_iv: "7007f758500328424369644e5e86b594"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.093164 20337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:55.107285 20337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:55.111982 20337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:55.112241 20337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:55.113173 20337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:55.114492 20337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:55.114564 20337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.114687 20337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:55.114741 20337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.143539 20337 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:46323
I20260809 13:09:55.143580 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:46323 every 8 connection(s)
I20260809 13:09:55.144773 20337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:55.148412 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 20337
I20260809 13:09:55.148594 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:55.149096 10966 external_mini_cluster.cc:1428] Setting key 47af3933fde196cb5c9d9c6b029047fa
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:55.167395 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.147213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:43715 (local address 127.10.181.190:44431)
0809 13:09:55.147381 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.147385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.148512 (+ 1127us) server_negotiation.cc:407] Connection header received
0809 13:09:55.150583 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.150589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.150677 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.150742 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.152904 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.154069 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.156238 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.156882 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.157015 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.161536 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.161542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.161546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.161580 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.164344 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.165023 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.165028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.165031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.165202 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.165629 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.165634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.165637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.166015 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.166170 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.166811 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.167046 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:09:55.168139 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.147648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44431 (local address 127.10.181.130:43715)
0809 13:09:55.148347 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:55.148388 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:55.150243 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:55.151166 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:55.151178 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:55.151891 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:55.152659 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:55.152685 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.154285 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.154293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:55.156066 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:55.156081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.157326 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.157335 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:55.157657 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.158481 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:55.158535 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:55.161280 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:55.164476 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:55.164483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:55.164496 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:55.164904 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:55.165294 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:55.165297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:55.165299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:55.165513 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:55.166122 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:55.166128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:55.166435 (+ 307us) client_negotiation.cc:769] Sending connection context
0809 13:09:55.166663 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 13:09:55.167043 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
I20260809 13:09:55.169680 20523 heartbeater.cc:347] Connected to a master server at 127.10.181.190:44431
I20260809 13:09:55.169987 20523 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:55.170706 20523 heartbeater.cc:511] Master 127.10.181.190:44431 requested a full tablet report, sending...
I20260809 13:09:55.172176 20029 ts_manager.cc:194] Registered new tserver with Master: bed508425fef405fabffcddd28aed01a (127.10.181.130:46323)
I20260809 13:09:55.173043 20029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:43715
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:09:55.206326 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:55.528659 20531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:55.528954 20531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:55.529099 20531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:55.536093 20531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:55.536208 20531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:55.536357 20531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:55.545732 20531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:55.546983 20531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:55.548280 20531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:55.560884 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:55.560832 20540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:55.560914 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:55.561662 20531 server_base.cc:1034] running on GCE node
I20260809 13:09:55.562198 20531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:55.563050 20531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:55.564253 20531 hybrid_clock.cc:648] HybridClock initialized: now 1786280995564195 us; error 53 us; skew 500 ppm
I20260809 13:09:55.564572 20531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.567528 20531 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:09:55.568902 20531 webserver.cc:460] Webserver started at http://127.10.181.131:34251/ using document root <none> and password file <none>
I20260809 13:09:55.569571 20531 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:55.569684 20531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:55.569940 20531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:55.572635 20531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "2b074986e4a440329358d80221b5eb6d"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "312dad733ab43b943053317118b481f0"
server_key_iv: "c0295ccab887233a57a51446549cef09"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.573354 20531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "2b074986e4a440329358d80221b5eb6d"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "312dad733ab43b943053317118b481f0"
server_key_iv: "c0295ccab887233a57a51446549cef09"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.578266 20531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 13:09:55.581558 20553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.582787 20531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:09:55.582940 20531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "2b074986e4a440329358d80221b5eb6d"
format_stamp: "Formatted at 2026-08-09 13:09:55 on dist-test-slave-42z9"
server_key: "312dad733ab43b943053317118b481f0"
server_key_iv: "c0295ccab887233a57a51446549cef09"
server_key_version: "encryptionkey@0"
I20260809 13:09:55.583127 20531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:55.605752 20531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:55.609321 20531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:55.609696 20531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:55.610628 20531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:55.611876 20531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:55.611954 20531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.612043 20531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:55.612085 20531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:55.641575 20531 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:46615
I20260809 13:09:55.641608 20730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:46615 every 8 connection(s)
I20260809 13:09:55.642769 20531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:55.645790 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 20531
I20260809 13:09:55.645949 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:55.646520 10966 external_mini_cluster.cc:1428] Setting key 1b078759109e11be1a791b5b329eabda
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:09:55.662899 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.645123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:46497 (local address 127.10.181.190:44431)
0809 13:09:55.645306 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.645311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.646634 (+ 1323us) server_negotiation.cc:407] Connection header received
0809 13:09:55.648951 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.648958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.649041 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.649136 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.650686 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.651434 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.652920 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.653322 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.653482 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.657539 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.657544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.657548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.657579 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.660039 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.660851 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.660856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.660859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.661014 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.661467 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.661473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.661475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.661792 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.661906 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.662499 (+ 593us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.662663 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:09:55.663683 20733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.645621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44431 (local address 127.10.181.131:46497)
0809 13:09:55.646444 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:55.646498 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:55.648699 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:55.649369 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:55.649380 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:55.649983 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:55.650513 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:55.650535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.651590 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.651596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:55.652778 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:55.652792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.653579 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.653587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:55.653858 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.654524 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:55.654555 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:55.657323 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:55.660163 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:55.660172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:55.660184 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:55.660716 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:55.661128 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:55.661132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:55.661134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:55.661310 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:55.661903 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:55.661909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:55.662188 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 13:09:55.662404 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 13:09:55.662741 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":311,"threads_started":1}
I20260809 13:09:55.664852 20731 heartbeater.cc:347] Connected to a master server at 127.10.181.190:44431
I20260809 13:09:55.665124 20731 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:55.665886 20731 heartbeater.cc:511] Master 127.10.181.190:44431 requested a full tablet report, sending...
I20260809 13:09:55.667177 20029 ts_manager.cc:194] Registered new tserver with Master: 2b074986e4a440329358d80221b5eb6d (127.10.181.131:46615)
I20260809 13:09:55.667980 20029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:46497
I20260809 13:09:55.673511 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:09:55.685977 20330 heartbeater.cc:503] Master 127.10.181.190:44431 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/test-admin.keytab.
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.704245 10966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 13:09:55.709090 20744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:55.706744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.10.181.129:36857)
0809 13:09:55.707385 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.707392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.707419 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:09:55.707483 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.707490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.707683 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.707822 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.708722 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36864: BlockingRecv error: recv got EOF from 127.0.0.1:36864 (error 108)
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":338,"threads_started":1}
W20260809 13:09:55.709156 10966 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc' of type FILE
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.710829 10966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.712127 10966 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:55.725939 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.713711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.10.181.129:36857)
0809 13:09:55.713899 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.713905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.714050 (+ 145us) server_negotiation.cc:407] Connection header received
0809 13:09:55.714195 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.714200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.714257 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.714326 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.715253 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.716202 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.717309 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.717894 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.717992 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.721224 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.721235 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.721248 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.721284 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.723689 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.724241 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.724246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.724248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.724365 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.724639 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.724642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.724644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.724957 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.725175 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.725393 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.725666 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 13:09:55.730940 20744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:55.729530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.10.181.129:36857)
0809 13:09:55.729707 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.729713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.729969 (+ 256us) server_negotiation.cc:407] Connection header received
0809 13:09:55.730069 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.730082 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.730132 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.730200 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.730743 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36884: BlockingRecv error: recv got EOF from 127.0.0.1:36884 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 13:09:55.731495 10966 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc)
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.732758 10966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.733916 10966 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:55.746268 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.735034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.10.181.129:36857)
0809 13:09:55.735209 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.735215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.735451 (+ 236us) server_negotiation.cc:407] Connection header received
0809 13:09:55.735579 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.735584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.735636 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.735710 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.736595 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.737279 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.738478 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.738900 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.739008 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.742200 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.742205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.742208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.742238 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.744337 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.744870 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.744873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.744875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.744982 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.745239 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.745242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.745244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.745517 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.745706 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.745908 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.746052 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 13:09:55.749747 20744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:09:55.748508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36906 (local address 127.10.181.129:36857)
0809 13:09:55.748685 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.748689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.748777 (+ 88us) server_negotiation.cc:407] Connection header received
0809 13:09:55.748879 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.748882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.748940 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.749006 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.749558 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36906: BlockingRecv error: recv got EOF from 127.0.0.1:36906 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 13:09:55.749962 10966 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786280946828509-10966-0/krb5kdc/krb5cc' of type FILE
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.751298 10966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:55.752461 10966 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:09:55.765230 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.753543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.10.181.129:36857)
0809 13:09:55.753715 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.753720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.753954 (+ 234us) server_negotiation.cc:407] Connection header received
0809 13:09:55.754149 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.754153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.754202 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.754271 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.755159 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.755818 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.757008 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.757420 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.757561 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.760572 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.760576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.760579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.760608 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.762850 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.763459 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.763463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.763465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.763627 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.763965 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.763968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.763970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.764225 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.764532 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.764777 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.764971 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:09:55.766693 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 20127
I20260809 13:09:55.776928 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.767325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.10.181.129:36857)
0809 13:09:55.767473 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.767479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.767750 (+ 271us) server_negotiation.cc:407] Connection header received
0809 13:09:55.767860 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.767863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.767912 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.767982 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.768819 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.769541 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.770597 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.771012 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.771098 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.772782 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.772786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.772789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.772811 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.774994 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.775525 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.775529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.775531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.775656 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.775899 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.775902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.775904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.776109 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.776295 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.776499 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.776638 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 13:09:55.777400 20315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:55.859642 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 20127
I20260809 13:09:55.883622 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 20337
Aug 09 13:09:55 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:09:55.898586 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:55.884536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.10.181.130:46323)
0809 13:09:55.885131 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:55.885139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:55.885165 (+ 26us) server_negotiation.cc:407] Connection header received
0809 13:09:55.885226 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:55.885232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:55.885465 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:55.885602 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:55.886572 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.887668 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.888822 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:55.889349 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:55.889471 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:55.893197 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:55.893208 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:55.893222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:55.893255 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:55.895850 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.896504 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.896511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.896513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.896641 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:55.896956 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:55.896959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:55.896961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:55.897301 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:55.897646 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:55.898084 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 13:09:55.898281 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":299,"threads_started":1}
I20260809 13:09:55.899824 20517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:55.980284 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 20337
I20260809 13:09:56.000718 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 20531
Aug 09 13:09:56 dist-test-slave-42z9 krb5kdc[19966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:09:56.016052 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:56.001678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.10.181.131:46615)
0809 13:09:56.002300 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:56.002307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:56.002336 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:09:56.002398 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:56.002404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:56.002616 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:56.002758 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:56.003845 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:56.004890 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:56.006270 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:56.006813 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:56.006930 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:56.010626 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:56.010638 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:56.010654 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:56.010693 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:56.013200 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:56.014051 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:56.014058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:56.014061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:56.014198 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:56.014564 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:56.014568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:56.014570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:56.014910 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:56.015222 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:56.015505 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 13:09:56.015765 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":304,"threads_started":1}
I20260809 13:09:56.017400 20724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:56.099725 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 20531
I20260809 13:09:56.123327 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 19983
I20260809 13:09:56.124506 20076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:09:56.226297 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 19983
2026-08-09T13:09:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4071 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:09:56 dist-test-slave-42z9 krb5kdc[20774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:09:56 dist-test-slave-42z9 krb5kdc[20774](info): set up 2 sockets
Aug 09 13:09:56 dist-test-slave-42z9 krb5kdc[20774](info): commencing operation
krb5kdc: starting...
W20260809 13:09:57.431810 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:09:57 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:09:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:09:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab.
Entry for principal oryx/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab.
I20260809 13:09:57.650665 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46275
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab
--principal=oryx/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:57.973276 20790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:57.973621 20790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:57.973765 20790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:57.980822 20790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:09:57.980940 20790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:57.981019 20790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:57.981092 20790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:09:57.981153 20790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:09:57.990852 20790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab
--principal=oryx/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:57.992116 20790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:57.993382 20790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:58.004436 20799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:58.004562 20802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:58.004524 20804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:58.006219 20790 server_base.cc:1034] running on GCE node
I20260809 13:09:58.006922 20790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:58.008248 20790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:58.009430 20790 hybrid_clock.cc:648] HybridClock initialized: now 1786280998009372 us; error 50 us; skew 500 ppm
I20260809 13:09:58.009788 20790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:58 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280998, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:58.012780 20790 init.cc:377] Logged in from keytab as oryx/127.10.181.190@KRBTEST.COM (short username oryx)
I20260809 13:09:58.014187 20790 webserver.cc:460] Webserver started at http://127.10.181.190:33465/ using document root <none> and password file <none>
I20260809 13:09:58.014822 20790 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:58.014919 20790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:58.015180 20790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:58.018086 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "1e05b48d43604a3483f7edd009414e60"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "8084bb76171ba23ff14f04cb20fd5653"
server_key_iv: "d250f0d757a3550966111bda44ae4fb1"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.018991 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "1e05b48d43604a3483f7edd009414e60"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "8084bb76171ba23ff14f04cb20fd5653"
server_key_iv: "d250f0d757a3550966111bda44ae4fb1"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.023962 20790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:09:58.027305 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:58.028535 20790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:09:58.028697 20790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "1e05b48d43604a3483f7edd009414e60"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "8084bb76171ba23ff14f04cb20fd5653"
server_key_iv: "d250f0d757a3550966111bda44ae4fb1"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.028864 20790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:58.044497 20790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:58.048555 20790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:58.048806 20790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:58.066056 20790 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46275
I20260809 13:09:58.066100 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46275 every 8 connection(s)
I20260809 13:09:58.067359 20790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:09:58.068472 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 20790
I20260809 13:09:58.068665 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:09:58.069291 10966 external_mini_cluster.cc:1428] Setting key aaae915c3d318815db652ee10ad77c79
I20260809 13:09:58.071475 20903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:09:58 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:09:58.084045 20903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Bootstrap starting.
I20260809 13:09:58.087818 20903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Neither blocks nor log segments found. Creating new log.
I20260809 13:09:58.089039 20903 log.cc:824] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Log is configured to *not* fsync() on all Append() calls
I20260809 13:09:58.091791 20903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: No bootstrap required, opened a new log
I20260809 13:09:58.092502 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:58.071208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.10.181.190:46275)
0809 13:09:58.071935 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:58.071945 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:58.071987 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:09:58.073602 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:58.073624 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:58.073992 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:58.074319 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:58.076128 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.077560 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.079388 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.080049 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.080195 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:58.084791 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:58.084802 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:58.084818 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:58.084867 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:58.088598 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:58.089244 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:58.089250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:58.089256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:58.089416 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:58.089813 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:58.089818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:58.089820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:58.090339 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:58.090628 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:58.091063 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 13:09:58.091456 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":308,"threads_started":1}
I20260809 13:09:58.099471 20903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } }
I20260809 13:09:58.099797 20903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:09:58.099881 20903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e05b48d43604a3483f7edd009414e60, State: Initialized, Role: FOLLOWER
I20260809 13:09:58.100370 20903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } }
I20260809 13:09:58.100590 20903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:09:58.100718 20903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:09:58.100873 20903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:09:58.103056 20903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } }
I20260809 13:09:58.103538 20903 leader_election.cc:302] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e05b48d43604a3483f7edd009414e60; no voters:
I20260809 13:09:58.104053 20903 leader_election.cc:288] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:09:58.104173 20910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:09:58.104776 20910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 LEADER]: Becoming Leader. State: Replica: 1e05b48d43604a3483f7edd009414e60, State: Running, Role: LEADER
I20260809 13:09:58.105274 20910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } }
I20260809 13:09:58.105949 20903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:09:58.108724 20913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e05b48d43604a3483f7edd009414e60. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } }
I20260809 13:09:58.109383 20913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:58.108882 20911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } }
I20260809 13:09:58.110098 20911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:09:58.110984 20923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:09:58.116375 20923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:09:58.129911 20923 catalog_manager.cc:1347] Generated new cluster ID: e6b64dcf847343ecae0b2ad70a3a59cd
I20260809 13:09:58.130081 20923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:09:58.157820 20923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:09:58.159794 20923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:09:58.167932 20923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Generated new TSK 0
I20260809 13:09:58.168846 20923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
I20260809 13:09:58.234937 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:58.607574 20938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:58.607904 20938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:58.608055 20938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:58.615180 20938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:58.615275 20938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:58.615391 20938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:09:58.624814 20938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:58.626138 20938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:58.627386 20938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:58.639794 20948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:58.639866 20945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:58.639839 20946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:58.640726 20938 server_base.cc:1034] running on GCE node
I20260809 13:09:58.641345 20938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:58.642264 20938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:58.643458 20938 hybrid_clock.cc:648] HybridClock initialized: now 1786280998643413 us; error 41 us; skew 500 ppm
I20260809 13:09:58.643754 20938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:58 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280998, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:58.646816 20938 init.cc:377] Logged in from keytab as oryx/127.10.181.129@KRBTEST.COM (short username oryx)
I20260809 13:09:58.648185 20938 webserver.cc:460] Webserver started at http://127.10.181.129:41953/ using document root <none> and password file <none>
I20260809 13:09:58.648837 20938 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:58.648936 20938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:58.649255 20938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:58.652163 20938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "a92694ab93e74051888e08196b620fcb"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "cac2c50c4022bd5f746824536afb155d"
server_key_iv: "ddf075e65656800afca92e25f0c6dc04"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.652892 20938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "a92694ab93e74051888e08196b620fcb"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "cac2c50c4022bd5f746824536afb155d"
server_key_iv: "ddf075e65656800afca92e25f0c6dc04"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.657784 20938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:09:58.661126 20965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:58.662382 20938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:58.662518 20938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "a92694ab93e74051888e08196b620fcb"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "cac2c50c4022bd5f746824536afb155d"
server_key_iv: "ddf075e65656800afca92e25f0c6dc04"
server_key_version: "encryptionkey@0"
I20260809 13:09:58.662688 20938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:58.681627 20938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:58.685545 20938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:58.685772 20938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:58.686648 20938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:58.687873 20938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:58.687944 20938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:58.688035 20938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:58.688076 20938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:58.717990 20938 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:32775
I20260809 13:09:58.718029 21134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:32775 every 8 connection(s)
I20260809 13:09:58.719200 20938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:09:58.727406 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 20938
I20260809 13:09:58.727624 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:09:58.728269 10966 external_mini_cluster.cc:1428] Setting key e0e8ef266a0897755e420e7940d13f77
Aug 09 13:09:58 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280998, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:09:58.738822 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:58.721619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:58567 (local address 127.10.181.190:46275)
0809 13:09:58.722067 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:58.722072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:58.723000 (+ 928us) server_negotiation.cc:407] Connection header received
0809 13:09:58.724766 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:58.724772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:58.724864 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:58.724961 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:58.726494 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.727373 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.728995 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.729512 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.729653 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:58.733201 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:58.733208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:58.733212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:58.733249 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:58.736019 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:58.736668 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:58.736673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:58.736677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:58.736821 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:58.737185 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:58.737189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:58.737192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:58.737537 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:58.737687 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:58.738274 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 13:09:58.738482 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":211,"threads_started":1}
I20260809 13:09:58.739557 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:58.722157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:58567)
0809 13:09:58.722843 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:58.722888 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:58.724506 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:58.725224 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:58.725232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:58.725766 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:58.726323 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:58.726340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.727547 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.727553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:58.728838 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:58.728854 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:58.729717 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:58.729722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:58.729924 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:58.730574 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:58.730601 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:58.733000 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:58.736147 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:58.736153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:58.736163 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:58.736553 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:58.736917 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:58.736919 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:58.736921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:58.737091 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:58.737645 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:58.737650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:58.737932 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 13:09:58.738145 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 13:09:58.738491 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":275,"threads_started":1}
I20260809 13:09:58.741024 21136 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:09:58.741357 21136 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:58.742277 21136 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:09:58.744298 20846 ts_manager.cc:194] Registered new tserver with Master: a92694ab93e74051888e08196b620fcb (127.10.181.129:32775)
I20260809 13:09:58.745819 20846 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:58567
WARNING: no policy specified for oryx/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
I20260809 13:09:58.782050 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:59.100278 21146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:59.100559 21146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:59.100701 21146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:59.107743 21146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:59.107851 21146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:59.107995 21146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:09:59.117314 21146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:59.118575 21146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:59.119889 21146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:59.132720 21151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:59.133222 21152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:59.134323 21154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:59.134841 21146 server_base.cc:1034] running on GCE node
I20260809 13:09:59.135414 21146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:59.136286 21146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:59.137516 21146 hybrid_clock.cc:648] HybridClock initialized: now 1786280999137462 us; error 37 us; skew 500 ppm
I20260809 13:09:59.137840 21146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:59 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280999, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:59.140909 21146 init.cc:377] Logged in from keytab as oryx/127.10.181.130@KRBTEST.COM (short username oryx)
I20260809 13:09:59.142374 21146 webserver.cc:460] Webserver started at http://127.10.181.130:34511/ using document root <none> and password file <none>
I20260809 13:09:59.143069 21146 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:59.143183 21146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:59.143476 21146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:59.146533 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "899d077a68fb4e1882ebcbfe1e78f502"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4ca9b388bdc307e9773bb4e797095bae"
server_key_iv: "a8fa9c384955d72c8b7b18802a250d67"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.147284 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "899d077a68fb4e1882ebcbfe1e78f502"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4ca9b388bdc307e9773bb4e797095bae"
server_key_iv: "a8fa9c384955d72c8b7b18802a250d67"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.160676 21146 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.014s sys 0.000s
I20260809 13:09:59.164897 21164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.166457 21146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:59.166631 21146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "899d077a68fb4e1882ebcbfe1e78f502"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4ca9b388bdc307e9773bb4e797095bae"
server_key_iv: "a8fa9c384955d72c8b7b18802a250d67"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.166860 21146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:59.185196 21146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:59.189010 21146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:59.189282 21146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:59.190229 21146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:59.191511 21146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:59.191584 21146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.191677 21146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:59.191720 21146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.221870 21146 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33365
I20260809 13:09:59.221906 21332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33365 every 8 connection(s)
I20260809 13:09:59.223109 21146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:09:59.232250 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 21146
I20260809 13:09:59.232430 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:09:59.232999 10966 external_mini_cluster.cc:1428] Setting key 668399a297e92dc35d119ecdbd237184
Aug 09 13:09:59 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280999, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:09:59.243086 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:59.225507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:57031 (local address 127.10.181.190:46275)
0809 13:09:59.225725 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:59.225731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:59.226908 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 13:09:59.228637 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:59.228641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:59.228711 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:59.228798 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:59.230232 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.230931 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.232710 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.233251 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.233347 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:59.237032 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:59.237038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:59.237042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:59.237081 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:59.240023 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:59.240838 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:59.240843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:59.240846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:59.240999 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:59.241375 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:59.241379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:59.241381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:59.241764 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:59.241915 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:59.242534 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 13:09:59.242747 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:09:59.243795 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:59.226052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:57031)
0809 13:09:59.226747 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:59.226786 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:59.228401 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:59.228983 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:59.228990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:59.229532 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:59.230083 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:59.230101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.231046 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.231049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:59.232571 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:59.232588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.233505 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.233514 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:59.233737 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:59.234408 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:59.234435 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:59.236800 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:59.240150 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:59.240156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:59.240168 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:59.240710 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:59.241092 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:59.241095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:59.241098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:59.241270 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:59.241873 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:59.241879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:59.242175 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 13:09:59.242390 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 13:09:59.242759 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":266,"threads_started":1}
I20260809 13:09:59.245106 21335 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:09:59.245421 21335 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:59.246285 21335 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:09:59.247758 20846 ts_manager.cc:194] Registered new tserver with Master: 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:09:59.248806 20846 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:57031
WARNING: no policy specified for oryx/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
I20260809 13:09:59.284252 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:09:59.601145 21341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:09:59.601405 21341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:09:59.601543 21341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:09:59.608402 21341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:09:59.608513 21341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:09:59.608670 21341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:09:59.617790 21341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:09:59.618952 21341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:09:59.620182 21341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:09:59.632567 21351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:59.632517 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:09:59.632593 21353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:09:59.633430 21341 server_base.cc:1034] running on GCE node
I20260809 13:09:59.634042 21341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:09:59.634879 21341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:09:59.636073 21341 hybrid_clock.cc:648] HybridClock initialized: now 1786280999636035 us; error 30 us; skew 500 ppm
I20260809 13:09:59.636380 21341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:09:59 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280999, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:09:59.642593 21341 init.cc:377] Logged in from keytab as oryx/127.10.181.131@KRBTEST.COM (short username oryx)
I20260809 13:09:59.643872 21341 webserver.cc:460] Webserver started at http://127.10.181.131:39295/ using document root <none> and password file <none>
I20260809 13:09:59.644475 21341 fs_manager.cc:359] Metadata directory not provided
I20260809 13:09:59.644578 21341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:09:59.644892 21341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:09:59.647706 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "b1051ac8c4824d28b26a7851e08efa4d"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4b7444efe480e3a8d4110bf189bde459"
server_key_iv: "186de32840e124bcebbf6d9f8accb60d"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.648418 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "b1051ac8c4824d28b26a7851e08efa4d"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4b7444efe480e3a8d4110bf189bde459"
server_key_iv: "186de32840e124bcebbf6d9f8accb60d"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.653227 21341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260809 13:09:59.656414 21376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.657672 21341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:09:59.657822 21341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "b1051ac8c4824d28b26a7851e08efa4d"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4b7444efe480e3a8d4110bf189bde459"
server_key_iv: "186de32840e124bcebbf6d9f8accb60d"
server_key_version: "encryptionkey@0"
I20260809 13:09:59.657986 21341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:09:59.678633 21341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:09:59.682343 21341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:09:59.682562 21341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:09:59.683405 21341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:09:59.684609 21341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:09:59.684684 21341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.684777 21341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:09:59.684819 21341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:09:59.712378 21341 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:33867
I20260809 13:09:59.712419 21537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:33867 every 8 connection(s)
I20260809 13:09:59.713686 21341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:09:59.723026 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 21341
I20260809 13:09:59.723206 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:09:59.723688 10966 external_mini_cluster.cc:1428] Setting key 615e6ec5ceaac982fe3b21dba397ce73
Aug 09 13:09:59 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280999, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:09:59.735913 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:59.716138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:58631 (local address 127.10.181.190:46275)
0809 13:09:59.716357 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:09:59.716363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:09:59.717539 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 13:09:59.720163 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:09:59.720168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:09:59.720238 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:09:59.720324 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:09:59.722247 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.723192 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.725014 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.725625 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.725755 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:59.729760 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:09:59.729767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:09:59.729772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:09:59.729813 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:09:59.732667 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:59.733594 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:59.733599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:59.733603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:59.733758 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:09:59.734204 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:09:59.734209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:09:59.734213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:09:59.734504 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:09:59.734637 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:09:59.735366 (+ 729us) server_negotiation.cc:299] Negotiation successful
0809 13:09:59.735584 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:09:59.736845 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:09:59.716596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:58631)
0809 13:09:59.717354 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 13:09:59.717401 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:09:59.719823 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:09:59.720605 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:09:59.720616 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:09:59.721322 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:09:59.722077 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:59.722100 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.723387 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.723392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:59.724881 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:09:59.724897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:09:59.725856 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:09:59.725864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:09:59.726137 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:09:59.726965 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:09:59.726992 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:09:59.729532 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:09:59.732817 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:59.732850 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:59.732865 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:59.733417 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:59.733882 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:09:59.733887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:09:59.733891 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:09:59.734087 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:09:59.734635 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:09:59.734642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:09:59.734989 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 13:09:59.735225 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 13:09:59.735710 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":264,"threads_started":1}
I20260809 13:09:59.739147 21538 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:09:59.739445 21538 heartbeater.cc:464] Registering TS with master...
I20260809 13:09:59.740242 21538 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:09:59.741865 20844 ts_manager.cc:194] Registered new tserver with Master: b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:09:59.742691 20844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:58631
I20260809 13:09:59.748624 21136 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
I20260809 13:09:59.750816 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.181.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.189.keytab.
Entry for principal oryx/127.10.181.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.189.keytab.
I20260809 13:09:59.803197 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:32987
--webserver_interface=127.10.181.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.189.keytab
--principal=oryx/127.10.181.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46275,127.10.181.189:32987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:00.126917 21545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:00.127286 21545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:00.127434 21545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:00.135103 21545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:00.135210 21545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:00.135282 21545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:00.135356 21545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:00.135421 21545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:00.145077 21545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46275,127.10.181.189:32987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.189.keytab
--principal=oryx/127.10.181.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:32987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.181.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
W20260809 13:10:00.145422 21545 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.181.190:46275,127.10.181.189:32987'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 13:10:00.146447 21545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:00.147718 21545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:00.158823 21553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.158862 21556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.158885 21554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:00.159789 21545 server_base.cc:1034] running on GCE node
I20260809 13:10:00.160557 21545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:00.162003 21545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:00.163245 21545 hybrid_clock.cc:648] HybridClock initialized: now 1786281000163173 us; error 43 us; skew 500 ppm
I20260809 13:10:00.163544 21545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:00.168991 21545 init.cc:377] Logged in from keytab as oryx/127.10.181.189@KRBTEST.COM (short username oryx)
I20260809 13:10:00.170452 21545 webserver.cc:460] Webserver started at http://127.10.181.189:45447/ using document root <none> and password file <none>
I20260809 13:10:00.171187 21545 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:00.171277 21545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:00.171546 21545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:00.174539 21545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/instance:
uuid: "b07c6c2c1db443509128f0c57796c05e"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "f06eae80e3d57badb0030292d5af0013"
server_key_iv: "7e95ee654f211032530852705d5520a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.175284 21545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal/instance:
uuid: "b07c6c2c1db443509128f0c57796c05e"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "f06eae80e3d57badb0030292d5af0013"
server_key_iv: "7e95ee654f211032530852705d5520a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.180542 21545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:10:00.183969 21573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:00.185309 21545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:00.185523 21545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "b07c6c2c1db443509128f0c57796c05e"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "f06eae80e3d57badb0030292d5af0013"
server_key_iv: "7e95ee654f211032530852705d5520a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.185776 21545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:00.202070 21545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:00.205978 21545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:00.206220 21545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:00.223191 21545 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.189:32987
I20260809 13:10:00.223237 21635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.189:32987 every 8 connection(s)
I20260809 13:10:00.224812 21545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
I20260809 13:10:00.229705 21638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:00.233181 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 21545
I20260809 13:10:00.239651 21638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:10:00.251246 21335 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.189@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:00.261562 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.243104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.10.181.190:46275)
0809 13:10:00.243572 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.243578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.244173 (+ 595us) server_negotiation.cc:407] Connection header received
0809 13:10:00.244236 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.244241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.244313 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.244399 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.245975 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.247057 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.248759 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.249266 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.249399 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.254366 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.254372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.254377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.254417 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.258093 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.259027 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.259033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.259036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.259177 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.259624 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.259629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.259632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.260116 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.260279 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.260846 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.261156 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
I20260809 13:10:00.262530 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.242705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40480)
0809 13:10:00.243871 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.243913 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.244059 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.244623 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.244630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.245487 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.245800 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.245813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.247226 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.247231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.248572 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.248585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.249533 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.249543 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.249736 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.250397 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.250429 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.254179 (+ 3750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.258271 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.258284 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.258291 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.258926 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.259311 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.259315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.259319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.259505 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.260267 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.260275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.260522 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.260731 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.261085 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":54,"spinlock_wait_cycles":21888,"thread_start_us":753,"threads_started":1}
I20260809 13:10:00.262559 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.235778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.10.181.189:32987)
0809 13:10:00.236700 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.236720 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.236774 (+ 54us) server_negotiation.cc:407] Connection header received
0809 13:10:00.239334 (+ 2560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.239363 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.239911 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.240352 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.242739 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.244673 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.246498 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.247511 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.247750 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.252853 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.252883 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.252901 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.252939 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.256924 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.257607 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.257615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.257623 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.257802 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.258276 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.258281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.258284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.258958 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.259257 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.260523 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.260967 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"spinlock_wait_cycles":16256,"thread_start_us":368,"threads_started":1}
I20260809 13:10:00.265750 21638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.189@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
I20260809 13:10:00.280357 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.266371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52430)
0809 13:10:00.266527 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.266544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.266646 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.267200 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.267204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.267605 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.267766 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.267777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.269106 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.269111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.270313 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.270325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.271244 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.271248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.271460 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.271965 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.271987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.274060 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.277715 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.277721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.277725 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.278210 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.278602 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.278607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.278610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.278761 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.279371 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.279386 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.279569 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.279710 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.280008 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260809 13:10:00.280532 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.266799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.10.181.189:32987)
0809 13:10:00.266961 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.266966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.266995 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:10:00.267030 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.267035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.267099 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.267180 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.267916 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.268978 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.270438 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.271090 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.271243 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.274262 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.274268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.274273 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.274309 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.277587 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.278326 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.278331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.278335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.278502 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.278872 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.278876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.278879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.279259 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.279423 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.279822 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.280181 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":37}
I20260809 13:10:00.285166 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Bootstrap starting.
I20260809 13:10:00.288290 21638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:00.289363 21638 log.cc:824] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:00.292006 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: No bootstrap required, opened a new log
I20260809 13:10:00.298839 21638 raft_consensus.cc:348] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } }
I20260809 13:10:00.299146 21638 raft_consensus.cc:374] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:00.299234 21638 raft_consensus.cc:729] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b07c6c2c1db443509128f0c57796c05e, State: Initialized, Role: FOLLOWER
I20260809 13:10:00.299770 21638 consensus_queue.cc:260] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } }
I20260809 13:10:00.300575 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } } }
I20260809 13:10:00.300869 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:00.301514 21638 sys_catalog.cc:564] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 13:10:00.309978 21664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:10:00.310130 21664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:10:00.312134 21545 master_runner.cc:428] Detected that this master b07c6c2c1db443509128f0c57796c05e is joining an existing cluster
I20260809 13:10:00.312214 21545 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.181.189:32987
I20260809 13:10:00.331156 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.315740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.10.181.190:46275)
0809 13:10:00.315933 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.315939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.316537 (+ 598us) server_negotiation.cc:407] Connection header received
0809 13:10:00.316739 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.316746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.316866 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.316957 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.318372 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.319077 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.321146 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.321825 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.321927 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.324603 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.324611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.324615 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.324653 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.328059 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.328806 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.328811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.328813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.328975 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.329468 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.329473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.329476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.329815 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.329959 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.330572 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.330819 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:10:00.331736 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.316758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.10.181.189:32987)
0809 13:10:00.316912 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.316917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.317504 (+ 587us) server_negotiation.cc:407] Connection header received
0809 13:10:00.317749 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.317755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.317825 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.317926 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.319179 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.320040 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.321861 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.322581 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.322691 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.324961 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.324967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.324972 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.325006 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.328387 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.329200 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.329205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.329209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.329368 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.329821 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.329826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.329830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.330225 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.330355 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.330848 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.331057 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260809 13:10:00.332233 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.315751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40486)
0809 13:10:00.316431 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.316450 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.316576 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.317197 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.317202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.317831 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.318239 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.318249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.319231 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.319245 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.320964 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.320979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.322246 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.322253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.322527 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.323170 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.323196 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.324378 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.328223 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.328227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.328230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.328662 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.329124 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.329128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.329131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.329301 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.330063 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.330066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.330280 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.331413 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.331732 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":108,"thread_start_us":333,"threads_started":1}
I20260809 13:10:00.333216 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.316594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52444)
0809 13:10:00.317381 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.317401 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.317606 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.317973 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.317977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.318677 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.319017 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.319030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.320241 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.320246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.321713 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.321728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.322915 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.322922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.323081 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.323646 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.323679 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.324769 (+ 1090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.328527 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.328533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.328537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.329063 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.329530 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.329534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.329538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.329699 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.330439 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.330443 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.330589 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.332167 (+ 1578us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.332799 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":1,"thread_start_us":473,"threads_started":1}
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280998, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.190@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.10.181.188@KRBTEST.COM; defaulting to no policy
I20260809 13:10:00.358124 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.339240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.10.181.189:32987)
0809 13:10:00.339408 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.339413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.342096 (+ 2683us) server_negotiation.cc:407] Connection header received
0809 13:10:00.342489 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.342495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.342579 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.342684 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.343986 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.345085 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.346795 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.347549 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.347680 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.351556 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.351563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.351567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.351612 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.355173 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.356051 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.356055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.356057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.356211 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.356639 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.356643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.356645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.356911 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.357042 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.357553 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.357755 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:10:00.358163 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.339145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52458)
0809 13:10:00.341963 (+ 2818us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.342012 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.342229 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.342749 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.342756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.343549 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.343819 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.343831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.345258 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.345262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.346628 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.346643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.347723 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.347729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.347915 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.348586 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.348606 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.351352 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.355313 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.355328 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.355337 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.355905 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.356319 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.356324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.356329 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.356518 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.357036 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.357044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.357208 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.357371 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.357731 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2600,"thread_start_us":2450,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.188@KRBTEST.COM" created.
I20260809 13:10:00.360844 20844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.181.189:32987: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b07c6c2c1db443509128f0c57796c05e"
member_type: NON_VOTER
last_known_addr {
host: "127.10.181.189"
port: 32987
}
attrs {
promote: true
}
}
dest_uuid: "1e05b48d43604a3483f7edd009414e60"
cas_config_opid_index: -1
I20260809 13:10:00.362490 20844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } }
I20260809 13:10:00.373692 21604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:00.375801 21681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b07c6c2c1db443509128f0c57796c05e (127.10.181.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.377655 21604 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 13:10:00.378748 21682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:00.381044 21681 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.181.189:32987
I20260809 13:10:00.381968 21545 master.cc:537] Master@127.10.181.189:32987 shutting down...
I20260809 13:10:00.382591 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.383278 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:00.401875 21604 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b07c6c2c1db443509128f0c57796c05e (127.10.181.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.406796 21693 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.407547 21693 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEARNER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.188.keytab.
Entry for principal oryx/127.10.181.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.188.keytab.
I20260809 13:10:00.407953 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.408351 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:00.408427 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e05b48d43604a3483f7edd009414e60. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.408761 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:00.409648 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:40825
--webserver_interface=127.10.181.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.188.keytab
--principal=oryx/127.10.181.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:10:00.423979 21545 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Raft consensus shutting down.
I20260809 13:10:00.424502 21545 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Raft consensus is shut down!
I20260809 13:10:00.424672 21545 tablet_replica.cc:330] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: stopping tablet replica
I20260809 13:10:00.432327 21545 master.cc:552] Master@127.10.181.189:32987 shutdown complete.
I20260809 13:10:00.432456 21545 master_runner.cc:305] Clearing existing system tablet
I20260809 13:10:00.432934 21545 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:00.433038 21545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:00.435335 21545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 13:10:00.436759 21697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:00.437081 21545 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:00.437355 21545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 13:10:00.437453 21545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "b07c6c2c1db443509128f0c57796c05e"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "f06eae80e3d57badb0030292d5af0013"
server_key_iv: "7e95ee654f211032530852705d5520a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.440876 21545 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:00.445554 21545 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 13:10:00.445710 21545 log.cc:1196] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 13:10:00.446110 21545 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Deleting consensus metadata
I20260809 13:10:00.446339 21545 master_runner.cc:315] Copying system tablet from 127.10.181.190:46275
I20260809 13:10:00.448846 21545 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: tablet copy: Beginning tablet copy session from remote peer at address 127.10.181.190:46275
I20260809 13:10:00.460899 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.449646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40502)
0809 13:10:00.450265 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.450280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.450422 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.450779 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.450784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.451213 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.451443 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.451453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.452461 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.452465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.453392 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.453405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.454132 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.454138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.454317 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.454864 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.454881 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.455617 (+ 736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.458472 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.458477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.458481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.458830 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.459187 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.459192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.459196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.459321 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.459829 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.459834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.459985 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.460135 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.460443 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":349,"threads_started":1}
I20260809 13:10:00.460894 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.449669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.10.181.190:46275)
0809 13:10:00.449861 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.449868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.450373 (+ 505us) server_negotiation.cc:407] Connection header received
0809 13:10:00.450573 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.450579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.450648 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.450735 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.451601 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.452315 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.453562 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.453979 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.454090 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.455851 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.455857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.455860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.455892 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.458343 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.458942 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.458947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.458951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.459083 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.459432 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.459437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.459441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.459709 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.459842 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.460242 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.460521 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:10:00.462074 20879 tablet_copy_service.cc:140] P 1e05b48d43604a3483f7edd009414e60: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b07c6c2c1db443509128f0c57796c05e ({username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:40502)
I20260809 13:10:00.462476 20879 tablet_copy_service.cc:161] P 1e05b48d43604a3483f7edd009414e60: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b07c6c2c1db443509128f0c57796c05e at {username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:40502: session id = b07c6c2c1db443509128f0c57796c05e-00000000000000000000000000000000
I20260809 13:10:00.466498 20879 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 13:10:00.468750 21545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 13:10:00.473706 21545 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: tablet copy: Starting download of 0 data blocks...
I20260809 13:10:00.474094 21545 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: tablet copy: Starting download of 1 WAL segments...
I20260809 13:10:00.477231 21545 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 13:10:00.480484 20879 tablet_copy_service.cc:342] P 1e05b48d43604a3483f7edd009414e60: Request end of tablet copy session b07c6c2c1db443509128f0c57796c05e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:40502
I20260809 13:10:00.480715 20879 tablet_copy_service.cc:433] P 1e05b48d43604a3483f7edd009414e60: ending tablet copy session b07c6c2c1db443509128f0c57796c05e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b07c6c2c1db443509128f0c57796c05e
I20260809 13:10:00.491621 21545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:00.494063 21712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.498788 21714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.505843 21720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:00.506353 21545 server_base.cc:1034] running on GCE node
I20260809 13:10:00.506687 21545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:00.507391 21545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:00.508589 21545 hybrid_clock.cc:648] HybridClock initialized: now 1786281000508527 us; error 36 us; skew 500 ppm
I20260809 13:10:00.508800 21545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:00.510712 21545 init.cc:377] Logged in from keytab as oryx/127.10.181.189@KRBTEST.COM (short username oryx)
I20260809 13:10:00.511809 21545 webserver.cc:460] Webserver started at http://127.10.181.189:38341/ using document root <none> and password file <none>
I20260809 13:10:00.512127 21545 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:00.512202 21545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:00.514675 21545 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 13:10:00.516251 21736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:00.516718 21545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 13:10:00.516855 21545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "b07c6c2c1db443509128f0c57796c05e"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "f06eae80e3d57badb0030292d5af0013"
server_key_iv: "7e95ee654f211032530852705d5520a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.516994 21545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:00.543056 21545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:00.546238 21545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:00.573766 21545 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.189:32987
I20260809 13:10:00.573786 21814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.189:32987 every 8 connection(s)
I20260809 13:10:00.574599 21545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
I20260809 13:10:00.580152 21815 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:10:00.584321 21815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Bootstrap starting.
I20260809 13:10:00.595593 21815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:10:00.596001 21815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Bootstrap complete.
I20260809 13:10:00.596784 21815 raft_consensus.cc:348] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } }
I20260809 13:10:00.596971 21815 raft_consensus.cc:729] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b07c6c2c1db443509128f0c57796c05e, State: Initialized, Role: LEARNER
I20260809 13:10:00.597211 21815 consensus_queue.cc:260] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } }
I20260809 13:10:00.597994 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.598274 21815 sys_catalog.cc:564] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:00.598398 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:00.605223 21836 catalog_manager.cc:1259] Loaded cluster ID: e6b64dcf847343ecae0b2ad70a3a59cd
I20260809 13:10:00.605366 21836 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: loading cluster ID for follower catalog manager: success
I20260809 13:10:00.608070 21836 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: acquiring CA information for follower catalog manager: success
I20260809 13:10:00.609920 21836 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:10:00.745044 21538 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
W20260809 13:10:00.853109 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:00.853379 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:00.853533 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:00.860564 21695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:00.860666 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:00.860736 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:00.860831 21695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:00.860888 21695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:00.862829 21681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.863262 21681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:00.870299 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.863071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52460)
0809 13:10:00.864140 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.864160 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.864289 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.864689 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.864692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.864726 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:00.864931 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.864940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.866802 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.866807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.868118 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.868134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.869699 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.869704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.869833 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.869877 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.869940 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.870040 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":862,"threads_started":1}
I20260809 13:10:00.870536 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.863334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.10.181.189:32987)
0809 13:10:00.864049 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.864057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.864243 (+ 186us) server_negotiation.cc:407] Connection header received
0809 13:10:00.864410 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.864415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.864513 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.864657 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:00.865069 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.866666 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.868268 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.869560 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.869745 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.869923 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.869973 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.870122 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":362,"threads_started":1}
I20260809 13:10:00.871438 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e05b48d43604a3483f7edd009414e60. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: NON_VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: true } } }
I20260809 13:10:00.871691 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:00.872522 21681 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: attempting to promote NON_VOTER b07c6c2c1db443509128f0c57796c05e to VOTER
I20260809 13:10:00.873481 21681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } }
I20260809 13:10:00.873823 21695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.188.keytab
--principal=oryx/127.10.181.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.181.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:00.875308 21780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 LEARNER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 13:10:00.875480 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:00.875887 21684 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 13:10:00.877143 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 13:10:00.878844 21681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b07c6c2c1db443509128f0c57796c05e (127.10.181.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } }
I20260809 13:10:00.879480 21780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b07c6c2c1db443509128f0c57796c05e (127.10.181.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } }
I20260809 13:10:00.882591 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } }
I20260809 13:10:00.882818 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:00.882735 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } }
I20260809 13:10:00.882946 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:00.883392 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } }
I20260809 13:10:00.883647 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
W20260809 13:10:00.889984 21860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.890053 21855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:00.890027 21856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:00.891110 21695 server_base.cc:1034] running on GCE node
I20260809 13:10:00.891866 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:00.893193 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:00.894382 21695 hybrid_clock.cc:648] HybridClock initialized: now 1786281000894327 us; error 42 us; skew 500 ppm
I20260809 13:10:00.894724 21695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:00.897730 21695 init.cc:377] Logged in from keytab as oryx/127.10.181.188@KRBTEST.COM (short username oryx)
I20260809 13:10:00.899169 21695 webserver.cc:460] Webserver started at http://127.10.181.188:37475/ using document root <none> and password file <none>
I20260809 13:10:00.899865 21695 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:00.899986 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:00.900286 21695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:00.903367 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/instance:
uuid: "91251e300a394715b13800187f1ae370"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "b108ab1aeb04a538a78c8ba0e6783da1"
server_key_iv: "355add9acd724567c13ff0bfe6543e47"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.904131 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal/instance:
uuid: "91251e300a394715b13800187f1ae370"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "b108ab1aeb04a538a78c8ba0e6783da1"
server_key_iv: "355add9acd724567c13ff0bfe6543e47"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.909200 21695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:10:00.912618 21874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:00.913915 21695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:00.914054 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "91251e300a394715b13800187f1ae370"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "b108ab1aeb04a538a78c8ba0e6783da1"
server_key_iv: "355add9acd724567c13ff0bfe6543e47"
server_key_version: "encryptionkey@0"
I20260809 13:10:00.914213 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:00.934087 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:00.937753 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:00.937983 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:00.954461 21695 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.188:40825
I20260809 13:10:00.954490 21951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.188:40825 every 8 connection(s)
I20260809 13:10:00.955786 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
I20260809 13:10:00.959290 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:00.962769 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 21695
I20260809 13:10:00.966777 21953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.188@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:00.986137 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.970443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.10.181.190:46275)
0809 13:10:00.970979 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.970987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.971424 (+ 437us) server_negotiation.cc:407] Connection header received
0809 13:10:00.971597 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.971601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.971672 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.971805 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.973171 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.974169 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.975947 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.976385 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.976478 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.979968 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.979975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.979980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.980018 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.983090 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.983913 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.983919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.983922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.984068 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.984466 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.984470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.984473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.984812 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.984958 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.985578 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.985802 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":307,"threads_started":1}
I20260809 13:10:00.987197 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.964976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.10.181.188:40825)
0809 13:10:00.965803 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.965816 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.965869 (+ 53us) server_negotiation.cc:407] Connection header received
0809 13:10:00.967706 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.967734 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.968231 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.968652 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.970056 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.971983 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.973621 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.974587 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.974731 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.979618 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.979626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.979638 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.979668 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.982957 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.983750 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.983757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.983759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.983991 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.984428 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.984433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.984436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.985072 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.985304 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.985728 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 13:10:00.986138 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":244,"server-negotiator.queue_time_us":579,"spinlock_wait_cycles":15744,"thread_start_us":356,"threads_started":1}
I20260809 13:10:00.987226 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.970475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40506)
0809 13:10:00.971295 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.971329 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.971480 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.971880 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.971887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.972693 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.973017 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.973030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.974320 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.974324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.975809 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.975826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.976624 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.976633 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.976817 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.977389 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.977427 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.979782 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.983213 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.983225 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.983237 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.983807 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.984170 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.984174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.984177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.984375 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.984923 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.984929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.985129 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.985473 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.985993 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":1,"spinlock_wait_cycles":22784,"thread_start_us":486,"threads_started":1}
I20260809 13:10:00.988686 21953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:10:00 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.188@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
I20260809 13:10:01.000316 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.989280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52470)
0809 13:10:00.989476 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:00.989495 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:00.989590 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:00.989992 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:00.989996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:00.990317 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:00.990442 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.990450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.991452 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.991458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.992338 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:00.992349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.993001 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.993007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:00.993176 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.993659 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:00.993674 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:00.995449 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:00.997987 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.997992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.997996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.998348 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.998738 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:00.998743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:00.998746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:00.998869 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:00.999367 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:00.999371 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:00.999517 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 13:10:00.999651 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 13:10:00.999921 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 13:10:01.000378 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:00.989501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.10.181.189:32987)
0809 13:10:00.989703 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:00.989709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:00.989730 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:10:00.989788 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:00.989793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:00.989851 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:00.989934 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:00.990587 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.991321 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.992470 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:00.992873 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:00.992977 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:00.995633 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:00.995638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:00.995641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:00.995671 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:00.997855 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.998452 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.998458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.998461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.998619 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:00.998968 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:00.998973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:00.998976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:00.999256 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:00.999397 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:00.999771 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.000029 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:10:01.001408 21953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281000, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.188@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
I20260809 13:10:01.012872 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.001981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.0.0.1:51562)
0809 13:10:01.002145 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.002162 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.002245 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.002467 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.002469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.002767 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.002908 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.002915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.003771 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.003774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.004548 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.004558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.005126 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.005130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.005278 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.005762 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.005778 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.007830 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.010652 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.010657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.010661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.010985 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.011321 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.011326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.011330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.011472 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.011936 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.011940 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.012054 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.012190 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.012437 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 13:10:01.012880 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.002145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.10.181.188:40825)
0809 13:10:01.002261 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.002265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.002298 (+ 33us) server_negotiation.cc:407] Connection header received
0809 13:10:01.002332 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.002335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.002384 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.002444 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.003015 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.003667 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.004656 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.005014 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.005102 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.007997 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.008001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.008005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.008032 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.010532 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.011090 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.011094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.011096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.011220 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.011570 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.011575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.011578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.011826 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.011949 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.012296 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.012579 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":34}
I20260809 13:10:01.017899 21953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Bootstrap starting.
I20260809 13:10:01.021140 21953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:01.022245 21953 log.cc:824] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:01.024716 21953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: No bootstrap required, opened a new log
I20260809 13:10:01.031591 21953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } }
I20260809 13:10:01.031929 21953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:01.032047 21953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91251e300a394715b13800187f1ae370, State: Initialized, Role: FOLLOWER
I20260809 13:10:01.032617 21953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } }
I20260809 13:10:01.033478 21964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } } }
I20260809 13:10:01.033788 21964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:01.034416 21953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:01.042085 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 20938
W20260809 13:10:01.042997 21978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:10:01.043128 21978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:10:01.047163 21695 master_runner.cc:428] Detected that this master 91251e300a394715b13800187f1ae370 is joining an existing cluster
I20260809 13:10:01.047283 21695 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.181.188:40825
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.129@KRBTEST.COM
I20260809 13:10:01.064242 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.043384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.10.181.129:32775)
0809 13:10:01.043985 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.043994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.044024 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:10:01.044094 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.044102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.044352 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.044516 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.045923 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.047366 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.048752 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.049591 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.049727 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.055538 (+ 5811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.055553 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.055569 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.055611 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.059845 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.060720 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.060729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.060732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.060897 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.061531 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.061536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.061539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.061979 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.062284 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.063536 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.063821 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":257,"threads_started":1}
I20260809 13:10:01.065702 21126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:01.075361 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.052044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.10.181.190:46275)
0809 13:10:01.052198 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.052203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.052502 (+ 299us) server_negotiation.cc:407] Connection header received
0809 13:10:01.052763 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.052768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.052865 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.052938 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.054518 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.055715 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.057831 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.058488 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.058588 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.061225 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.061233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.061238 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.061275 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.064880 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.070032 (+ 5152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.070038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.070041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.070232 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.072954 (+ 2722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.072960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.072963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.073501 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.073649 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.074734 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.074976 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 13:10:01.077764 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.051664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40510)
0809 13:10:01.052416 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.052433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.052611 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.053154 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.053159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.053934 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.054342 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.054354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.055905 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.055910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.057648 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.057663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.058906 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.058913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.059090 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.059732 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.059757 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.060982 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.065073 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.065078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.065083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.069929 (+ 4846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.072218 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.072222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.072226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.072455 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.073754 (+ 1299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.073759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.074008 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.075624 (+ 1616us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.075942 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":1,"thread_start_us":440,"threads_started":1}
I20260809 13:10:01.087139 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.052550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52482)
0809 13:10:01.061371 (+ 8821us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.061389 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.061705 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.062300 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.062304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.062936 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.063267 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.063282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.064906 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.064910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.070369 (+ 5459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.070386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.071709 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.071715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.071904 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.072497 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.072521 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.074167 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.083023 (+ 8856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.083028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.083039 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.083800 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.084505 (+ 705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.084518 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.084521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.084818 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.085990 (+ 1172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.085996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.086214 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.086350 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.086662 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8691,"thread_start_us":8506,"threads_started":1}
I20260809 13:10:01.088054 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.052883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.10.181.189:32987)
0809 13:10:01.053069 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.053075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.061472 (+ 8397us) server_negotiation.cc:407] Connection header received
0809 13:10:01.061859 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.061865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.061941 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.062035 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.063580 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.064716 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.070525 (+ 5809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.071317 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.071407 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.076513 (+ 5106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.076520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.076525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.076574 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.082785 (+ 6211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.084035 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.084044 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.084051 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.084282 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.085050 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.085059 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.085066 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.085612 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.085848 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.087470 (+ 1622us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.087726 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280998, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.190@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
I20260809 13:10:01.104959 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.061432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.10.181.188:40825)
0809 13:10:01.061609 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.061614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.068495 (+ 6881us) server_negotiation.cc:407] Connection header received
0809 13:10:01.068844 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.068849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.068916 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.068976 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.071022 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.072141 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.074135 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.074892 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.074983 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.091450 (+ 16467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.091456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.091461 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.091496 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.095217 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.100082 (+ 4865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.100088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.100102 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.100309 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.102069 (+ 1760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.102073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.102084 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.102779 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.102956 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.103913 (+ 957us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.104367 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:10:01.106580 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.061583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.0.0.1:51570)
0809 13:10:01.068399 (+ 6816us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.068418 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.068725 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.069151 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.069156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.070567 (+ 1411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.070877 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.070889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.072337 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.072342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.073995 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.074007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.075256 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.075261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.075684 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.089854 (+ 14170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.089876 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.090767 (+ 891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.097156 (+ 6389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.097164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.097167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.097932 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.100540 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.100545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.100552 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.101888 (+ 1336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.103080 (+ 1192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.103089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.103387 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.105333 (+ 1946us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.105813 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6685,"thread_start_us":6926,"threads_started":1}
I20260809 13:10:01.108887 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.090949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.0.0.1:51572)
0809 13:10:01.091098 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.091116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.091213 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.092292 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.092297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.092794 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.092978 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.092991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.094968 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.094974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.096295 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.096311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.097336 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.097343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.097695 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.098477 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.098504 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.101362 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.105806 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.105811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.105814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.106373 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.107117 (+ 744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.107122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.107125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.107300 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.107971 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.107976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.108149 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.108276 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.108540 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 13:10:01.110083 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.091281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.10.181.188:40825)
0809 13:10:01.091895 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.091901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.091922 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:10:01.092010 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.092019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.092102 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.093121 (+ 1019us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.093477 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.094785 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.096488 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.097134 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.098046 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.101670 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.101677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.101682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.101714 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.105645 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.106543 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.106548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.106551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.106985 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.107449 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.107453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.107456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.107812 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.109480 (+ 1668us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.109591 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.109756 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":324,"threads_started":1}
I20260809 13:10:01.111476 20844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.181.188:40825: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "91251e300a394715b13800187f1ae370"
member_type: NON_VOTER
last_known_addr {
host: "127.10.181.188"
port: 40825
}
attrs {
promote: true
}
}
dest_uuid: "1e05b48d43604a3483f7edd009414e60"
cas_config_opid_index: 6
I20260809 13:10:01.112154 20844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } }
I20260809 13:10:01.114404 21915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:01.114529 21780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 13:10:01.115270 21684 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 13:10:01.118011 21684 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 91251e300a394715b13800187f1ae370 (127.10.181.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.118808 21915 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 13:10:01.120868 21684 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.181.188:40825
I20260809 13:10:01.121419 21695 master.cc:537] Master@127.10.181.188:40825 shutting down...
I20260809 13:10:01.121934 21682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:01.124382 21681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.124691 21681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:01.124642 21780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 91251e300a394715b13800187f1ae370 (127.10.181.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.125072 21681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.125305 21681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
W20260809 13:10:01.126104 20818 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 13:10:01.127846 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.128085 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:01.135046 21695 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:01.135592 21695 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:01.135874 21695 tablet_replica.cc:330] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: stopping tablet replica
W20260809 13:10:01.136912 20818 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 -> Peer 91251e300a394715b13800187f1ae370 (127.10.181.188:40825): Couldn't send request to peer 91251e300a394715b13800187f1ae370. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 13:10:01.143656 21695 master.cc:552] Master@127.10.181.188:40825 shutdown complete.
I20260809 13:10:01.143739 21695 master_runner.cc:305] Clearing existing system tablet
I20260809 13:10:01.144127 21695 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:01.144236 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:01.153120 21695 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.008s sys 0.002s
I20260809 13:10:01.157840 21998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:01.158187 21695 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:01.158452 21695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:10:01.158553 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "91251e300a394715b13800187f1ae370"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "b108ab1aeb04a538a78c8ba0e6783da1"
server_key_iv: "355add9acd724567c13ff0bfe6543e47"
server_key_version: "encryptionkey@0"
I20260809 13:10:01.163594 21695 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:01.170593 21695 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 13:10:01.170744 21695 log.cc:1196] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 13:10:01.171192 21695 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Deleting consensus metadata
I20260809 13:10:01.171464 21695 master_runner.cc:315] Copying system tablet from 127.10.181.190:46275
I20260809 13:10:01.173921 21695 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: tablet copy: Beginning tablet copy session from remote peer at address 127.10.181.190:46275
I20260809 13:10:01.192871 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.174693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40520)
0809 13:10:01.178397 (+ 3704us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.178416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.178645 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.179193 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.179198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.179766 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.180067 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.180081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.181634 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.181640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.183189 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.183205 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.184213 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.184220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.184402 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.185102 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.185129 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.186249 (+ 1120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.190048 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.190054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.190058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.190549 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.190954 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.190959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.190963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.191145 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.191776 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.191781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.191981 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.192142 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.192447 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3553,"thread_start_us":4346,"threads_started":1}
I20260809 13:10:01.192885 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.174768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.10.181.190:46275)
0809 13:10:01.174943 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.174948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.178517 (+ 3569us) server_negotiation.cc:407] Connection header received
0809 13:10:01.178808 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.178814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.178899 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.178980 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.180246 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.181478 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.183345 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.184033 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.184163 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.186484 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.186492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.186498 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.186540 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.189914 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.190665 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.190671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.190675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.190829 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.191270 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.191275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.191278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.191648 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.191797 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.192267 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.192521 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:10:01.193853 20879 tablet_copy_service.cc:140] P 1e05b48d43604a3483f7edd009414e60: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 91251e300a394715b13800187f1ae370 ({username='oryx', principal='oryx/127.10.181.188@KRBTEST.COM'} at 127.0.0.1:40520)
I20260809 13:10:01.194028 20879 tablet_copy_service.cc:161] P 1e05b48d43604a3483f7edd009414e60: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 91251e300a394715b13800187f1ae370 at {username='oryx', principal='oryx/127.10.181.188@KRBTEST.COM'} at 127.0.0.1:40520: session id = 91251e300a394715b13800187f1ae370-00000000000000000000000000000000
I20260809 13:10:01.197625 20879 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 13:10:01.199676 21695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 13:10:01.206593 21695 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: tablet copy: Starting download of 0 data blocks...
I20260809 13:10:01.206887 21695 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: tablet copy: Starting download of 1 WAL segments...
I20260809 13:10:01.209350 21695 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 13:10:01.213497 20879 tablet_copy_service.cc:342] P 1e05b48d43604a3483f7edd009414e60: Request end of tablet copy session 91251e300a394715b13800187f1ae370-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.181.188@KRBTEST.COM'} at 127.0.0.1:40520
I20260809 13:10:01.213637 20879 tablet_copy_service.cc:433] P 1e05b48d43604a3483f7edd009414e60: ending tablet copy session 91251e300a394715b13800187f1ae370-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 91251e300a394715b13800187f1ae370
I20260809 13:10:01.224437 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:01.227030 22008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:01.237020 22009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:01.237596 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 20938
I20260809 13:10:01.253683 21695 server_base.cc:1034] running on GCE node
W20260809 13:10:01.254845 22011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:01.255317 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:01.256095 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:01.257388 21695 hybrid_clock.cc:648] HybridClock initialized: now 1786281001257337 us; error 28 us; skew 500 ppm
I20260809 13:10:01.257652 21695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281001, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:01.259879 21695 init.cc:377] Logged in from keytab as oryx/127.10.181.188@KRBTEST.COM (short username oryx)
I20260809 13:10:01.264817 21695 webserver.cc:460] Webserver started at http://127.10.181.188:33553/ using document root <none> and password file <none>
I20260809 13:10:01.265218 21695 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:01.265322 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:01.268333 21695 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 13:10:01.270279 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:01.270777 21695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 13:10:01.270902 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "91251e300a394715b13800187f1ae370"
format_stamp: "Formatted at 2026-08-09 13:10:00 on dist-test-slave-42z9"
server_key: "b108ab1aeb04a538a78c8ba0e6783da1"
server_key_iv: "355add9acd724567c13ff0bfe6543e47"
server_key_version: "encryptionkey@0"
I20260809 13:10:01.271080 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:01.276742 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:32775
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:46275
--webserver_port=41953
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:10:01.303287 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:01.307263 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:01.342152 21695 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.188:40825
I20260809 13:10:01.342314 22098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.188:40825 every 8 connection(s)
I20260809 13:10:01.343343 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
I20260809 13:10:01.348757 22101 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:10:01.353119 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Bootstrap starting.
I20260809 13:10:01.372354 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:10:01.372876 22101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: Bootstrap complete.
I20260809 13:10:01.374047 22101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } }
I20260809 13:10:01.374346 22101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 91251e300a394715b13800187f1ae370, State: Initialized, Role: LEARNER
I20260809 13:10:01.374606 22101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } }
I20260809 13:10:01.375445 22106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.375810 22101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:01.375989 22106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:01.389612 22127 catalog_manager.cc:1259] Loaded cluster ID: e6b64dcf847343ecae0b2ad70a3a59cd
I20260809 13:10:01.389725 22127 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: loading cluster ID for follower catalog manager: success
I20260809 13:10:01.393239 22127 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: acquiring CA information for follower catalog manager: success
I20260809 13:10:01.395761 22127 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 13:10:01.647792 22032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:01.648070 22032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:01.648211 22032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:01.655370 22032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:01.655484 22032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:01.655637 22032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:01.665783 22032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:32775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=41953
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:01.667047 22032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:01.668380 22032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:01.682288 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:01.682488 22142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:01.682883 22136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:01.682912 22032 server_base.cc:1034] running on GCE node
I20260809 13:10:01.683646 22032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:01.684819 22032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:01.686015 22032 hybrid_clock.cc:648] HybridClock initialized: now 1786281001685965 us; error 42 us; skew 500 ppm
I20260809 13:10:01.686347 22032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281001, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:01.689777 22032 init.cc:377] Logged in from keytab as oryx/127.10.181.129@KRBTEST.COM (short username oryx)
I20260809 13:10:01.691229 22032 webserver.cc:460] Webserver started at http://127.10.181.129:41953/ using document root <none> and password file <none>
I20260809 13:10:01.691890 22032 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:01.692010 22032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:01.697590 22032 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:01.700484 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:01.701874 22032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:01.702023 22032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "a92694ab93e74051888e08196b620fcb"
format_stamp: "Formatted at 2026-08-09 13:09:58 on dist-test-slave-42z9"
server_key: "cac2c50c4022bd5f746824536afb155d"
server_key_iv: "ddf075e65656800afca92e25f0c6dc04"
server_key_version: "encryptionkey@0"
I20260809 13:10:01.702993 22032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:01.722317 22032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:01.726444 22032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:01.726730 22032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:01.727770 22032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:01.729213 22032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:01.729306 22032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:01.729419 22032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:01.729517 22032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 13:10:01.750274 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.742874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.0.0.1:51584)
0809 13:10:01.743388 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.743417 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.743569 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.744087 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.744092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.744121 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:01.744306 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.744321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.746305 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.746310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.747866 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.747880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.749434 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.749471 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.749658 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.749694 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.749780 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.749933 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260809 13:10:01.750296 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.743000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.10.181.188:40825)
0809 13:10:01.743689 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.743696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.743719 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:10:01.743801 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.743807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.743912 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.744046 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:01.744487 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.746133 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.748019 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.749246 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.749404 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.749633 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.749831 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.749973 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":404,"threads_started":1}
I20260809 13:10:01.751590 22064 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260809 13:10:01.752296 22106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e05b48d43604a3483f7edd009414e60. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:01.752586 22106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: LEARNER
I20260809 13:10:01.772547 22032 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:32775
I20260809 13:10:01.772598 22307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:32775 every 8 connection(s)
I20260809 13:10:01.774153 22032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:01.774726 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 22032
I20260809 13:10:01.775082 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 21146
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281001, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.130@KRBTEST.COM
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281001, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
Aug 09 13:10:01 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281001, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:01.815263 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.788193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53909 (local address 127.10.181.188:40825)
0809 13:10:01.788388 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.788394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.790673 (+ 2279us) server_negotiation.cc:407] Connection header received
0809 13:10:01.792345 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.792352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.792459 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.792533 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.795598 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.796747 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.801525 (+ 4778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.802205 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.802322 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.807231 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.807237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.807243 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.807285 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.810699 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.812069 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.812075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.812078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.812253 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.812915 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.812920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.812923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.813292 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.813461 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.814666 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.814907 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:10:01.817685 22325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.787921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.10.181.129:53909)
0809 13:10:01.790555 (+ 2634us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.790574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.791981 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.793135 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.793149 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.794306 (+ 1157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.795302 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.795333 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.796978 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.796984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.801346 (+ 4362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.801363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.802661 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.802668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.802996 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.803675 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.803714 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.806981 (+ 3267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.811092 (+ 4111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.811101 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.811119 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.811926 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.812443 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.812448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.812451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.812771 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.813657 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.813669 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.814134 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.815573 (+ 1439us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.816152 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2495,"thread_start_us":2752,"threads_started":1}
I20260809 13:10:01.817975 22319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.776589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.10.181.130:33365)
0809 13:10:01.790146 (+ 13557us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.790155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.790188 (+ 33us) server_negotiation.cc:407] Connection header received
0809 13:10:01.790263 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.790272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.790548 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.794251 (+ 3703us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.794571 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.796254 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.800788 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.801742 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.801864 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.809561 (+ 7697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.809577 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.809604 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.809651 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.813703 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.814566 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.814577 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.814580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.814743 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.815190 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.815194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.815196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.815629 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.815930 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.817220 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.817556 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13387,"thread_start_us":13238,"threads_started":1}
I20260809 13:10:01.819465 21327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:01.819548 22310 heartbeater.cc:347] Connected to a master server at 127.10.181.188:40825
I20260809 13:10:01.819885 22310 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:01.820909 22310 heartbeater.cc:511] Master 127.10.181.188:40825 requested a full tablet report, sending...
I20260809 13:10:01.822171 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.778226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:33415)
0809 13:10:01.787394 (+ 9168us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.787468 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.802044 (+ 14576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.805803 (+ 3759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.805808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.806505 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.806786 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.806799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.809088 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.809094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.810603 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.810619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.811834 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.811844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.812035 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.812612 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.812634 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.815050 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.819321 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.819326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.819329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.819857 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.820318 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.820327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.820529 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.821190 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.821195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.821327 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.821466 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.821762 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8788,"mutex_wait_us":2,"thread_start_us":8159,"threads_started":1}
I20260809 13:10:01.822999 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.791247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:52269 (local address 127.10.181.189:32987)
0809 13:10:01.804077 (+ 12830us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.804083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.804122 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:10:01.804187 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.804192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.804285 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.804400 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.805935 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.807036 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.809426 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.810178 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.810297 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.814459 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.814465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.814470 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.814510 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.819237 (+ 4727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820234 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.820241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.820245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.820414 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820864 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.820870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.820873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.821214 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.821359 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.822356 (+ 997us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.822580 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12692,"thread_start_us":14522,"threads_started":1}
I20260809 13:10:01.823256 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.777551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:33415 (local address 127.10.181.190:46275)
0809 13:10:01.805323 (+ 27772us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:01.805330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:01.805350 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:10:01.805418 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:01.805423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:01.805530 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:01.807012 (+ 1482us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:01.807336 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.808902 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.810777 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.811362 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.811480 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.815572 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:01.815580 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:01.815585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:01.815625 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:01.819182 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820037 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.820042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.820045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.820185 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820687 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:01.820692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:01.820694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:01.821050 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:01.822502 (+ 1452us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:01.822638 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 13:10:01.822860 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27629,"thread_start_us":28064,"threads_started":1}
I20260809 13:10:01.823599 22050 ts_manager.cc:194] Registered new tserver with Master: a92694ab93e74051888e08196b620fcb (127.10.181.129:32775)
I20260809 13:10:01.823716 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:01.791096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.10.181.129:52269)
0809 13:10:01.802011 (+ 10915us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:01.802030 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:01.802157 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:01.804609 (+ 2452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:01.804614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:01.805265 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:01.805667 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.805680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.807252 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.807258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.809251 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:01.809266 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:01.810548 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:01.810555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:01.810731 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:01.811325 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:01.811349 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:01.814225 (+ 2876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:01.819420 (+ 5195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.819425 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.819429 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.820079 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.820549 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:01.820554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:01.820558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:01.820732 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:01.821425 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:01.821430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:01.821628 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 13:10:01.821766 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:10:01.822066 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10789,"thread_start_us":15170,"threads_started":1}
I20260809 13:10:01.825047 22314 heartbeater.cc:347] Connected to a master server at 127.10.181.189:32987
I20260809 13:10:01.825186 22314 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:01.825538 22308 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:10:01.825682 22308 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:01.825886 22314 heartbeater.cc:511] Master 127.10.181.189:32987 requested a full tablet report, sending...
I20260809 13:10:01.826082 22308 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:10:01.827729 21763 ts_manager.cc:194] Registered new tserver with Master: a92694ab93e74051888e08196b620fcb (127.10.181.129:32775)
I20260809 13:10:01.828881 20844 ts_manager.cc:194] Re-registered known tserver with Master: a92694ab93e74051888e08196b620fcb (127.10.181.129:32775)
I20260809 13:10:01.830092 20844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:33415
I20260809 13:10:01.907725 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 21146
I20260809 13:10:01.933303 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:33365
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:46275
--webserver_port=34511
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:10:02.153092 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: NON_VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: true } } }
I20260809 13:10:02.153479 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:02.155256 22333 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: attempting to promote NON_VOTER 91251e300a394715b13800187f1ae370 to VOTER
I20260809 13:10:02.156013 22333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } }
I20260809 13:10:02.158617 22064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 LEARNER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 13:10:02.158818 21780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Refusing update from remote peer 1e05b48d43604a3483f7edd009414e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 13:10:02.159339 22332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 13:10:02.159828 22333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 13:10:02.163368 22333 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 91251e300a394715b13800187f1ae370 (127.10.181.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.164151 21780 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 91251e300a394715b13800187f1ae370 (127.10.181.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.165863 22064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 91251e300a394715b13800187f1ae370 (127.10.181.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.167577 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.167812 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:02.168140 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.168378 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:02.168619 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.168805 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:02.169293 22339 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.169905 22339 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:02.170639 22106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e05b48d43604a3483f7edd009414e60" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:02.171032 22106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 13:10:02.274781 22331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:02.275048 22331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:02.275185 22331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:02.282433 22331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:02.282544 22331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:02.282692 22331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:02.292363 22331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:33365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=34511
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:02.293745 22331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:02.295066 22331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:02.308269 22347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:02.308502 22350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:02.308701 22345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:02.309310 22331 server_base.cc:1034] running on GCE node
I20260809 13:10:02.309916 22331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:02.310920 22331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:02.312131 22331 hybrid_clock.cc:648] HybridClock initialized: now 1786281002312083 us; error 37 us; skew 500 ppm
I20260809 13:10:02.312449 22331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:02.315528 22331 init.cc:377] Logged in from keytab as oryx/127.10.181.130@KRBTEST.COM (short username oryx)
I20260809 13:10:02.316949 22331 webserver.cc:460] Webserver started at http://127.10.181.130:34511/ using document root <none> and password file <none>
I20260809 13:10:02.317780 22331 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:02.317920 22331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:02.323447 22331 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:10:02.326218 22358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:02.327564 22331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:02.327716 22331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "899d077a68fb4e1882ebcbfe1e78f502"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4ca9b388bdc307e9773bb4e797095bae"
server_key_iv: "a8fa9c384955d72c8b7b18802a250d67"
server_key_version: "encryptionkey@0"
I20260809 13:10:02.328699 22331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:02.344734 22331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:02.348819 22331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:02.349079 22331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:02.350065 22331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:02.351392 22331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:02.351495 22331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:02.351611 22331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:02.351691 22331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:02.382957 22331 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33365
I20260809 13:10:02.383006 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33365 every 8 connection(s)
I20260809 13:10:02.384600 22331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:02.389429 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 22331
I20260809 13:10:02.389792 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 21341
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786280997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.131@KRBTEST.COM
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
I20260809 13:10:02.433591 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.387742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:44949 (local address 127.10.181.190:46275)
0809 13:10:02.389088 (+ 1346us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:02.389094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:02.390338 (+ 1244us) server_negotiation.cc:407] Connection header received
0809 13:10:02.401281 (+ 10943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:02.401287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:02.401381 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:02.401462 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:02.406137 (+ 4675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.407664 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.412298 (+ 4634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.413300 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.413435 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.421595 (+ 8160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:02.421604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:02.421609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:02.421653 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:02.425214 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.429549 (+ 4335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.429554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.429557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.429737 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.430994 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.430999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.431002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.431372 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:02.431510 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:02.432799 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0809 13:10:02.433036 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":1705,"threads_started":1}
I20260809 13:10:02.434072 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.392712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.10.181.131:33867)
0809 13:10:02.405980 (+ 13268us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:02.405990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:02.409854 (+ 3864us) server_negotiation.cc:407] Connection header received
0809 13:10:02.410279 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:02.410291 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:02.410617 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:02.410810 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:02.413220 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.414885 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.416877 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.418648 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.418763 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.425477 (+ 6714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:02.425492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:02.425511 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:02.425560 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:02.429275 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.430382 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.430392 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.430395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.430553 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.431014 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.431019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.431022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.432188 (+ 1166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:02.432543 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:02.433364 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 13:10:02.433703 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13092,"thread_start_us":12839,"threads_started":1}
I20260809 13:10:02.435854 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.388435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:44949)
0809 13:10:02.390159 (+ 1724us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:02.390222 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:02.400729 (+ 10507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:02.402004 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:02.402014 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:02.402690 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:02.403602 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.403628 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.407964 (+ 4336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.407970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.411340 (+ 3370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.411359 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.414013 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.414023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.414586 (+ 563us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.415541 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:02.415572 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:02.421295 (+ 5723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:02.427168 (+ 5873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.427176 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.427194 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.427846 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.430543 (+ 2697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.430548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.430551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.430850 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.431679 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:02.431692 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:02.432206 (+ 514us) client_negotiation.cc:769] Sending connection context
0809 13:10:02.433859 (+ 1653us) client_negotiation.cc:240] Negotiation successful
0809 13:10:02.434422 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"thread_start_us":1123,"threads_started":1}
I20260809 13:10:02.436584 21529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:02.440492 22535 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:10:02.440783 22535 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:02.441742 22535 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:10:02.443516 20844 ts_manager.cc:194] Re-registered known tserver with Master: 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:10:02.444677 20844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:44949
I20260809 13:10:02.449364 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.390759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.10.181.130:54295)
0809 13:10:02.411946 (+ 21187us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:02.411966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:02.412082 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:02.417635 (+ 5553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:02.417640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:02.418288 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:02.418612 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.418626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.422564 (+ 3938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.423513 (+ 949us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.427012 (+ 3499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.427029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.429049 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.429057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.429236 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.429965 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:02.429987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:02.433033 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:02.443235 (+ 10202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.443241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.443245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.444052 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.444517 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.444522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.444525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.444700 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.448294 (+ 3594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:02.448299 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:02.448446 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 13:10:02.448580 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 13:10:02.448868 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21044,"thread_start_us":20863,"threads_started":1}
I20260809 13:10:02.450747 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.391008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:54295 (local address 127.10.181.188:40825)
0809 13:10:02.417067 (+ 26059us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:02.417073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:02.417093 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:10:02.417164 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:02.417169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:02.417249 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:02.419744 (+ 2495us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:02.419944 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.421086 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.428026 (+ 6940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.428654 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.428779 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.433291 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:02.433298 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:02.433304 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:02.433344 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:02.443079 (+ 9735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.444206 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.444213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.444217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.444375 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.447759 (+ 3384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.447764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.447768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.448125 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:02.448273 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:02.450112 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0809 13:10:02.450361 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25917,"thread_start_us":24339,"threads_started":1}
I20260809 13:10:02.451527 22536 heartbeater.cc:347] Connected to a master server at 127.10.181.188:40825
I20260809 13:10:02.451682 22536 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:02.452088 22536 heartbeater.cc:511] Master 127.10.181.188:40825 requested a full tablet report, sending...
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
I20260809 13:10:02.453265 22050 ts_manager.cc:194] Registered new tserver with Master: 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:10:02.460835 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.418278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.10.181.130:53921)
0809 13:10:02.427660 (+ 9382us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:02.427680 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:02.427798 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:02.430346 (+ 2548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:02.430351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:02.431785 (+ 1434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:02.434601 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.434618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.444820 (+ 10202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.444825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.449091 (+ 4266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:02.449106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.450323 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.450330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:02.450508 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.451106 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:02.451131 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:02.454129 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:02.457992 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.457998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.458002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.458553 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.458972 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:02.458978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:02.458982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:02.459160 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:02.459837 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:02.459843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:02.460007 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 13:10:02.460145 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:10:02.460447 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9237,"thread_start_us":9583,"threads_started":1}
I20260809 13:10:02.460901 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:02.412297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:53921 (local address 127.10.181.189:32987)
0809 13:10:02.429851 (+ 17554us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:02.429858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:02.429879 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:10:02.429952 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:02.429957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:02.430035 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:02.430147 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:02.435676 (+ 5529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.436799 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.449268 (+ 12469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:02.449930 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:02.450057 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:02.454360 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:02.454367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:02.454373 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:02.454424 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:02.457848 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.458684 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.458690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.458694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.458846 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:02.459277 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:02.459281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:02.459285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:02.459708 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:02.459853 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:02.460268 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 13:10:02.460496 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17412,"thread_start_us":21984,"threads_started":1}
I20260809 13:10:02.462042 22537 heartbeater.cc:347] Connected to a master server at 127.10.181.189:32987
I20260809 13:10:02.462193 22537 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:02.462491 22537 heartbeater.cc:511] Master 127.10.181.189:32987 requested a full tablet report, sending...
I20260809 13:10:02.463672 21763 ts_manager.cc:194] Registered new tserver with Master: 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:10:02.531332 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 21341
I20260809 13:10:02.559926 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:33867
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:46275
--webserver_port=39295
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:10:02.832424 22308 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
W20260809 13:10:02.913185 22559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:02.913472 22559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:02.913617 22559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:02.920598 22559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:02.920689 22559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:02.920823 22559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:02.930065 22559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=39295
--tserver_master_addrs=127.10.181.190:46275,127.10.181.189:32987,127.10.181.188:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:02.931303 22559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:02.932637 22559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:02.945165 22572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:02.945218 22570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:02.945166 22575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:02.945801 22559 server_base.cc:1034] running on GCE node
I20260809 13:10:02.946364 22559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:02.947216 22559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:02.948423 22559 hybrid_clock.cc:648] HybridClock initialized: now 1786281002948371 us; error 40 us; skew 500 ppm
I20260809 13:10:02.948717 22559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:02 dist-test-slave-42z9 krb5kdc[20774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:02.951608 22559 init.cc:377] Logged in from keytab as oryx/127.10.181.131@KRBTEST.COM (short username oryx)
I20260809 13:10:02.952953 22559 webserver.cc:460] Webserver started at http://127.10.181.131:39295/ using document root <none> and password file <none>
I20260809 13:10:02.953616 22559 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:02.953754 22559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:02.958700 22559 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:10:02.961283 22584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:02.962580 22559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:02.962731 22559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "b1051ac8c4824d28b26a7851e08efa4d"
format_stamp: "Formatted at 2026-08-09 13:09:59 on dist-test-slave-42z9"
server_key: "4b7444efe480e3a8d4110bf189bde459"
server_key_iv: "186de32840e124bcebbf6d9f8accb60d"
server_key_version: "encryptionkey@0"
I20260809 13:10:02.963650 22559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:02.985137 22559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:02.989274 22559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:02.989567 22559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:02.990476 22559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:02.991788 22559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:02.991894 22559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:02.992002 22559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:02.992071 22559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:03.022854 22559 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:33867
I20260809 13:10:03.022925 22754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:33867 every 8 connection(s)
I20260809 13:10:03.024204 22559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:03.029011 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 22559
Aug 09 13:10:03 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.188@KRBTEST.COM
Aug 09 13:10:03 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
Aug 09 13:10:03 dist-test-slave-42z9 krb5kdc[20774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281002, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.189@KRBTEST.COM
I20260809 13:10:03.063966 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.029999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:58781 (local address 127.10.181.189:32987)
0809 13:10:03.044798 (+ 14799us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.044802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.044817 (+ 15us) server_negotiation.cc:407] Connection header received
0809 13:10:03.044865 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.044869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.044959 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.045057 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.048550 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.049639 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.051586 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.052164 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.052265 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.056506 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.056513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.056519 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.056559 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.060428 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.061275 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.061281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.061285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.061534 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.062055 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.062061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.062064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.062399 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.062523 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.063138 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.063594 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14692,"thread_start_us":14580,"threads_started":1}
I20260809 13:10:03.064081 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.028365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:54885)
0809 13:10:03.029009 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.029030 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.030690 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.032713 (+ 2023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.032719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.033667 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:03.034576 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.034603 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.041296 (+ 6693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.041301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.042988 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.043005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.044209 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.044219 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.044515 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.045324 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:03.045355 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:03.049085 (+ 3730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:03.053715 (+ 4630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.053724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.053740 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.054496 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.056227 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.056232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.056236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.056507 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.061144 (+ 4637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:03.061155 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:03.061656 (+ 501us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.061992 (+ 336us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.062560 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":1073,"threads_started":1}
I20260809 13:10:03.064903 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.029835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.10.181.131:58781)
0809 13:10:03.044208 (+ 14373us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.044229 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.044361 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.046781 (+ 2420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.046786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.047942 (+ 1156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:03.048356 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.048370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.049819 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.049824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.051414 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.051432 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.052555 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.052561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.052739 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.053252 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:03.053275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:03.056023 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:03.060602 (+ 4579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.060607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.060611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.061134 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.061677 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.061681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.061685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.061907 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.062634 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:03.062639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:03.062797 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.064232 (+ 1435us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.064505 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14215,"thread_start_us":14056,"threads_started":1}
I20260809 13:10:03.066099 22771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.028531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:54885 (local address 127.10.181.190:46275)
0809 13:10:03.029496 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.029501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.029521 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:10:03.031237 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.031243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.031339 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.034826 (+ 3487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.035042 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.036252 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.043216 (+ 6964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.043928 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.044952 (+ 1024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.049334 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.049341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.049346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.049384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.053543 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.054869 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.054875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.054878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.055052 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.057627 (+ 2575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.057633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.057637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.060396 (+ 2759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.064906 (+ 4510us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.065523 (+ 617us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.065747 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":723,"threads_started":1}
I20260809 13:10:03.066390 22759 heartbeater.cc:347] Connected to a master server at 127.10.181.189:32987
I20260809 13:10:03.066736 22759 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:03.067268 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.026937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:57615 (local address 127.10.181.188:40825)
0809 13:10:03.027412 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.027417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.028141 (+ 724us) server_negotiation.cc:407] Connection header received
0809 13:10:03.031506 (+ 3365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.031513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.031620 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.031706 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.037344 (+ 5638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.038581 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.044174 (+ 5593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.044787 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.044890 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.055277 (+ 10387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.055284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.055290 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.055335 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.059233 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.060565 (+ 1332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.060572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.060576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.060743 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.065319 (+ 4576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.065325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.065328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.065792 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.065939 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.066537 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.066877 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":241,"threads_started":1}
I20260809 13:10:03.067723 22759 heartbeater.cc:511] Master 127.10.181.189:32987 requested a full tablet report, sending...
I20260809 13:10:03.068439 22768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.027300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.10.181.131:57615)
0809 13:10:03.027982 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.028018 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.031065 (+ 3047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.032281 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.032293 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.036847 (+ 4554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:03.037149 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.037162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.038830 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.038836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.041070 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.041086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.045180 (+ 4094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.045186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.045370 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.045913 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:03.045930 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:03.054702 (+ 8772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:03.059427 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.059432 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.059436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.059959 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.060890 (+ 931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:03.060895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:03.060899 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:03.065170 (+ 4271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:03.066021 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:03.066026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:03.066194 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.067577 (+ 1383us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.067877 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":253,"threads_started":1}
I20260809 13:10:03.069191 22756 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46275
I20260809 13:10:03.069331 22756 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:03.069845 22756 heartbeater.cc:511] Master 127.10.181.190:46275 requested a full tablet report, sending...
I20260809 13:10:03.071631 21763 ts_manager.cc:194] Registered new tserver with Master: b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:10:03.072471 22757 heartbeater.cc:347] Connected to a master server at 127.10.181.188:40825
I20260809 13:10:03.072599 22757 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:03.072768 20844 ts_manager.cc:194] Re-registered known tserver with Master: b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:10:03.072985 22757 heartbeater.cc:511] Master 127.10.181.188:40825 requested a full tablet report, sending...
I20260809 13:10:03.073945 20844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:54885
I20260809 13:10:03.074158 22050 ts_manager.cc:194] Registered new tserver with Master: b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:10:03.078099 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.047490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.10.181.190:46275)
0809 13:10:03.055206 (+ 7716us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.055212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.055230 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:03.055278 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.055283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.055346 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.058005 (+ 2659us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.058288 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.059731 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.061671 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.062555 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.063252 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.069711 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.069717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.069722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.069760 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.073856 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.074776 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.074781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.074785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.074935 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.075345 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.075351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.075354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.075777 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.077049 (+ 1272us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.077200 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.077679 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7589,"thread_start_us":17560,"threads_started":1}
I20260809 13:10:03.087097 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.053598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.10.181.189:32987)
0809 13:10:03.054788 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.054793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.067549 (+ 12756us) server_negotiation.cc:407] Connection header received
0809 13:10:03.067806 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.067811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.067882 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.067959 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.070164 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.071383 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.075892 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.076518 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.076612 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.081339 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.081345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.081349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.081388 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.084435 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.085127 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.085132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.085134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.085273 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.085683 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.085688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.085690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.085949 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.086183 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.086527 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.086740 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067,"thread_start_us":908,"threads_started":1}
I20260809 13:10:03.082921 20844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 13:10:03.093006 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.077103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.10.181.188:40825)
0809 13:10:03.077258 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.077263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.077644 (+ 381us) server_negotiation.cc:407] Connection header received
0809 13:10:03.077782 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.077787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.077871 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.077973 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.080101 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.080998 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.082711 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.083327 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.083426 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.087016 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.087022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.087027 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.087065 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.090394 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.091170 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.091174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.091176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.091304 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.091715 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.091720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.091723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.092027 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.092223 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.092538 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.092713 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 13:10:03.093237 20844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:03.132682 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.122487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33365 (local address 127.0.0.1:34392)
0809 13:10:03.123199 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.123216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.123323 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.124267 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.124272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.124297 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:03.125004 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.125022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.127643 (+ 2621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.127648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.130022 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.130037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.131925 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.131931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.132085 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.132117 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.132204 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.132293 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":3,"thread_start_us":407,"threads_started":1}
I20260809 13:10:03.133194 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.122637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.10.181.130:33365)
0809 13:10:03.123570 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.123581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.123616 (+ 35us) server_negotiation.cc:407] Connection header received
0809 13:10:03.123745 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.123755 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.124050 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.124292 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:03.125274 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.127035 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.130191 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.131718 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.131893 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.132080 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.132265 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.132441 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":591,"threads_started":1}
I20260809 13:10:03.135936 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.110506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:32775 (local address 127.0.0.1:39012)
0809 13:10:03.122171 (+ 11665us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.122194 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.122350 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.127073 (+ 4723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.127078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.127102 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:03.127423 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.127435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.128887 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.128892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.130855 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.130869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.135023 (+ 4154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.135031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.135190 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.135222 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.135324 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.135459 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11548,"mutex_wait_us":40,"thread_start_us":11051,"threads_started":1}
I20260809 13:10:03.136204 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.110773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.10.181.129:32775)
0809 13:10:03.126580 (+ 15807us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.126587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.126609 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:10:03.126692 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.126698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.126860 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.127048 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:03.127630 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.128752 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.133255 (+ 4503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.134718 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.134954 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.135133 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.135419 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.135752 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15663,"thread_start_us":16453,"threads_started":1}
I20260809 13:10:03.137313 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.110077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33867 (local address 127.0.0.1:46842)
0809 13:10:03.121282 (+ 11205us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.121304 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.121470 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.128488 (+ 7018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.128494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.128521 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:03.128979 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.128992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.130754 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.130763 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.133081 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.133097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.136551 (+ 3454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.136560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.136725 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.136756 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.136822 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.136939 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11057,"thread_start_us":11794,"threads_started":1}
I20260809 13:10:03.138420 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.110767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.10.181.131:33867)
0809 13:10:03.127922 (+ 17155us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.127929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.127962 (+ 33us) server_negotiation.cc:407] Connection header received
0809 13:10:03.128054 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.128062 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.128273 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.128496 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:03.129196 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.130594 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.133597 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.134770 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.137679 (+ 2909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.137838 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.137918 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.138072 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16978,"thread_start_us":16784,"threads_started":1}
I20260809 13:10:03.138649 22436 tablet_service.cc:1467] Processing CreateTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a (DEFAULT_TABLE table=test-table [id=705701c7aef54f72af021bf8d9bb532c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:03.140213 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a (DEFAULT_TABLE table=test-table [id=705701c7aef54f72af021bf8d9bb532c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:03.141317 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2776238da46c4d94b6a7c3e2e1dc043a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:03.142071 22436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2776238da46c4d94b6a7c3e2e1dc043a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:03.141772 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a (DEFAULT_TABLE table=test-table [id=705701c7aef54f72af021bf8d9bb532c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:03.142840 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2776238da46c4d94b6a7c3e2e1dc043a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:03.161684 22803 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Bootstrap starting.
I20260809 13:10:03.164006 22805 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Bootstrap starting.
I20260809 13:10:03.164878 22803 tablet_bootstrap.cc:654] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:03.166067 22803 log.cc:824] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:03.168375 22806 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Bootstrap starting.
I20260809 13:10:03.168623 22805 tablet_bootstrap.cc:654] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:03.170863 22805 log.cc:824] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:03.172438 22803 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: No bootstrap required, opened a new log
I20260809 13:10:03.172869 22803 ts_tablet_manager.cc:1397] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 13:10:03.173487 22806 tablet_bootstrap.cc:654] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:03.174834 22805 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: No bootstrap required, opened a new log
I20260809 13:10:03.175205 22805 ts_tablet_manager.cc:1397] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 13:10:03.175343 22806 log.cc:824] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:03.179970 22806 tablet_bootstrap.cc:492] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: No bootstrap required, opened a new log
I20260809 13:10:03.180357 22806 ts_tablet_manager.cc:1397] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260809 13:10:03.185647 22805 raft_consensus.cc:348] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.186012 22805 raft_consensus.cc:374] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:03.186107 22805 raft_consensus.cc:729] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92694ab93e74051888e08196b620fcb, State: Initialized, Role: FOLLOWER
I20260809 13:10:03.186726 22805 consensus_queue.cc:260] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.188597 22803 raft_consensus.cc:348] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.188913 22805 ts_tablet_manager.cc:1428] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 13:10:03.189024 22803 raft_consensus.cc:374] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:03.189121 22803 raft_consensus.cc:729] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1051ac8c4824d28b26a7851e08efa4d, State: Initialized, Role: FOLLOWER
I20260809 13:10:03.189724 22803 consensus_queue.cc:260] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.193132 22806 raft_consensus.cc:348] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.193655 22806 raft_consensus.cc:374] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:03.193759 22806 raft_consensus.cc:729] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899d077a68fb4e1882ebcbfe1e78f502, State: Initialized, Role: FOLLOWER
I20260809 13:10:03.194339 22806 consensus_queue.cc:260] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.195340 22756 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
I20260809 13:10:03.196023 22803 ts_tablet_manager.cc:1428] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Time spent starting tablet: real 0.023s user 0.017s sys 0.005s
I20260809 13:10:03.199057 22535 heartbeater.cc:503] Master 127.10.181.190:46275 was elected leader, sending a full tablet report...
I20260809 13:10:03.199743 22806 ts_tablet_manager.cc:1428] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Time spent starting tablet: real 0.019s user 0.012s sys 0.009s
W20260809 13:10:03.276489 22761 tablet.cc:2369] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:10:03.300419 22316 tablet.cc:2369] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:10:03.375650 22810 raft_consensus.cc:482] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:03.376191 22810 raft_consensus.cc:504] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.378334 22810 leader_election.cc:288] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867), 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:10:03.386286 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.379708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54059 (local address 127.10.181.130:33365)
0809 13:10:03.379907 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.379913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.380219 (+ 306us) server_negotiation.cc:407] Connection header received
0809 13:10:03.380348 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.380353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.380430 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.380525 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:03.381067 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.382021 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.384418 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.385237 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.385380 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.385482 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.385892 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.386038 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:10:03.386348 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.379536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33365 (local address 127.10.181.129:54059)
0809 13:10:03.380113 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.380127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.380234 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.380556 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.380560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.380597 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:03.380928 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.380939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.382204 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.382209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.384271 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.384282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.385469 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.385476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.385641 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.385675 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.385826 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.385956 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":25,"thread_start_us":303,"threads_started":1}
I20260809 13:10:03.386515 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.378656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:37683 (local address 127.10.181.131:33867)
0809 13:10:03.378838 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.378842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.379542 (+ 700us) server_negotiation.cc:407] Connection header received
0809 13:10:03.379756 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.379761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.379846 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.379933 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:03.380568 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.381509 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.384253 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.385545 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.385693 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.385762 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.386150 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.386285 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:10:03.387044 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.378741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33867 (local address 127.10.181.129:37683)
0809 13:10:03.379408 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:03.379428 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:03.379599 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:03.380010 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:03.380016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:03.380047 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:03.380406 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.380419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.381642 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.381648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.384045 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:03.384060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.385777 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.385784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:03.385944 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.385972 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 13:10:03.386683 (+ 711us) client_negotiation.cc:240] Negotiation successful
0809 13:10:03.386779 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":87,"thread_start_us":345,"threads_started":1}
I20260809 13:10:03.387581 22465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2776238da46c4d94b6a7c3e2e1dc043a" candidate_uuid: "a92694ab93e74051888e08196b620fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899d077a68fb4e1882ebcbfe1e78f502" is_pre_election: true
I20260809 13:10:03.388227 22465 raft_consensus.cc:2454] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92694ab93e74051888e08196b620fcb in term 0.
I20260809 13:10:03.388474 22685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2776238da46c4d94b6a7c3e2e1dc043a" candidate_uuid: "a92694ab93e74051888e08196b620fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1051ac8c4824d28b26a7851e08efa4d" is_pre_election: true
I20260809 13:10:03.388906 22157 leader_election.cc:302] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899d077a68fb4e1882ebcbfe1e78f502, a92694ab93e74051888e08196b620fcb; no voters:
I20260809 13:10:03.388983 22685 raft_consensus.cc:2454] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92694ab93e74051888e08196b620fcb in term 0.
I20260809 13:10:03.389287 22810 raft_consensus.cc:2781] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:03.389460 22810 raft_consensus.cc:482] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:03.389597 22810 raft_consensus.cc:3037] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:03.392242 22810 raft_consensus.cc:504] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:03.393091 22810 leader_election.cc:288] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [CANDIDATE]: Term 1 election: Requested vote from peers b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867), 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365)
I20260809 13:10:03.393352 22685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2776238da46c4d94b6a7c3e2e1dc043a" candidate_uuid: "a92694ab93e74051888e08196b620fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1051ac8c4824d28b26a7851e08efa4d"
I20260809 13:10:03.393565 22685 raft_consensus.cc:3037] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:03.393545 22465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2776238da46c4d94b6a7c3e2e1dc043a" candidate_uuid: "a92694ab93e74051888e08196b620fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "899d077a68fb4e1882ebcbfe1e78f502"
I20260809 13:10:03.393757 22465 raft_consensus.cc:3037] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:03.396063 22465 raft_consensus.cc:2454] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92694ab93e74051888e08196b620fcb in term 1.
I20260809 13:10:03.396418 22157 leader_election.cc:302] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899d077a68fb4e1882ebcbfe1e78f502, a92694ab93e74051888e08196b620fcb; no voters:
I20260809 13:10:03.396734 22685 raft_consensus.cc:2454] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92694ab93e74051888e08196b620fcb in term 1.
I20260809 13:10:03.396763 22810 raft_consensus.cc:2781] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:03.397518 22810 raft_consensus.cc:686] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 1 LEADER]: Becoming Leader. State: Replica: a92694ab93e74051888e08196b620fcb, State: Running, Role: LEADER
I20260809 13:10:03.398204 22810 consensus_queue.cc:237] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
W20260809 13:10:03.399497 22541 tablet.cc:2369] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:10:03.403620 20846 catalog_manager.cc:5495] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb reported cstate change: term changed from 0 to 1, leader changed from <none> to a92694ab93e74051888e08196b620fcb (127.10.181.129). New cstate: current_term: 1 leader_uuid: "a92694ab93e74051888e08196b620fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } health_report { overall_health: UNKNOWN } } }
I20260809 13:10:03.491048 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.485875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.10.181.129:32775)
0809 13:10:03.486165 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.486170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.486188 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:03.486378 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.486382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.486474 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.486603 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:03.487025 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.487737 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.488987 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.489381 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.489527 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.489942 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:03.490497 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:03.490611 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.490649 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.490731 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 13:10:03.500041 22685 raft_consensus.cc:1261] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Refusing update from remote peer a92694ab93e74051888e08196b620fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:03.500742 22465 raft_consensus.cc:1261] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Refusing update from remote peer a92694ab93e74051888e08196b620fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:03.502173 22822 consensus_queue.cc:1035] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 13:10:03.502632 22810 consensus_queue.cc:1035] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:03.544502 20844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:03.544793 20844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:03.551484 20844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e05b48d43604a3483f7edd009414e60: Sending DeleteTablet for 3 replicas of tablet 2776238da46c4d94b6a7c3e2e1dc043a
I20260809 13:10:03.553195 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:03 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:46842
I20260809 13:10:03.554020 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:03 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:39012
I20260809 13:10:03.554262 22839 tablet_replica.cc:330] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: stopping tablet replica
I20260809 13:10:03.554768 22839 raft_consensus.cc:2229] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:03.556137 22839 raft_consensus.cc:2258] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:03.556401 22436 tablet_service.cc:1514] Processing DeleteTablet for tablet 2776238da46c4d94b6a7c3e2e1dc043a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:03 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34392
I20260809 13:10:03.558446 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 20790
I20260809 13:10:03.559516 20892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:03.559904 22839 ts_tablet_manager.cc:1905] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:03.569439 22842 tablet_replica.cc:330] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: stopping tablet replica
I20260809 13:10:03.570449 22842 raft_consensus.cc:2229] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:03.574090 22842 raft_consensus.cc:2258] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:03.578226 22842 ts_tablet_manager.cc:1905] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:03.583339 22839 ts_tablet_manager.cc:1918] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:03.583520 22839 log.cc:1196] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/2776238da46c4d94b6a7c3e2e1dc043a
I20260809 13:10:03.584088 22839 ts_tablet_manager.cc:1939] T 2776238da46c4d94b6a7c3e2e1dc043a P b1051ac8c4824d28b26a7851e08efa4d: Deleting consensus metadata
I20260809 13:10:03.587474 22841 tablet_replica.cc:330] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: stopping tablet replica
I20260809 13:10:03.587949 22841 raft_consensus.cc:2229] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:03.588688 22841 raft_consensus.cc:2258] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:03.591985 22841 ts_tablet_manager.cc:1905] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:03.595521 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.558399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.10.181.189:32987)
0809 13:10:03.558595 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.558601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.558619 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:03.558675 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.558679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.558756 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.558855 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.569969 (+ 11114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.571431 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.581638 (+ 10207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.582372 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.582511 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.587630 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.587639 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.587644 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.587691 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.591568 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.592575 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.592582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.592586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.592734 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.593188 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.593193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.593197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.593589 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.593824 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.594391 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.594627 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 13:10:03.602639 22842 ts_tablet_manager.cc:1918] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:03.602833 22842 log.cc:1196] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/2776238da46c4d94b6a7c3e2e1dc043a
I20260809 13:10:03.603403 22842 ts_tablet_manager.cc:1939] T 2776238da46c4d94b6a7c3e2e1dc043a P 899d077a68fb4e1882ebcbfe1e78f502: Deleting consensus metadata
I20260809 13:10:03.609333 22841 ts_tablet_manager.cc:1918] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:03.609540 22841 log.cc:1196] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/2776238da46c4d94b6a7c3e2e1dc043a
I20260809 13:10:03.610176 22841 ts_tablet_manager.cc:1939] T 2776238da46c4d94b6a7c3e2e1dc043a P a92694ab93e74051888e08196b620fcb: Deleting consensus metadata
I20260809 13:10:03.614811 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.558153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.10.181.188:40825)
0809 13:10:03.558319 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.558324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.596816 (+ 38492us) server_negotiation.cc:407] Connection header received
0809 13:10:03.597066 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.597084 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.597171 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.597243 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.599052 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.600205 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.602276 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.602920 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.603016 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.607181 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.607188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.607193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.607235 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.611133 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.611938 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.611945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.611949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.612166 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.612663 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.612669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.612672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.613029 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.613689 (+ 660us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.614238 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.614452 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:10:03.718581 20820 catalog_manager.cc:4841] TS a92694ab93e74051888e08196b620fcb (127.10.181.129:32775): tablet 2776238da46c4d94b6a7c3e2e1dc043a (table test-table [id=705701c7aef54f72af021bf8d9bb532c]) successfully deleted
I20260809 13:10:03.719496 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 20790
W20260809 13:10:03.749010 22359 connection.cc:535] server connection from 127.0.0.1:34392 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 13:10:03.769984 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.755260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.10.181.188:40825)
0809 13:10:03.755530 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.755536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.755660 (+ 124us) server_negotiation.cc:407] Connection header received
0809 13:10:03.755821 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.755826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.755888 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.755972 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.757733 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.758849 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.760362 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.760898 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.761029 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.764105 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.764110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.764113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.764144 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.766676 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.767577 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.767582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.767586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.767724 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.768127 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.768133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.768136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.768568 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.768733 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.769255 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.769651 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 13:10:03.770188 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:03.754393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.10.181.189:32987)
0809 13:10:03.754611 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:03.754619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:03.754729 (+ 110us) server_negotiation.cc:407] Connection header received
0809 13:10:03.755088 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:03.755094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:03.755189 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:03.755293 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:03.757389 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.758267 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.759832 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:03.760463 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:03.760590 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:03.763388 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:03.763393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:03.763396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:03.763434 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:03.766644 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.767865 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.767871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.767874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.768198 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:03.768578 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:03.768583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:03.768586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:03.768912 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:03.769140 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:03.769516 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 13:10:03.769733 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 13:10:04.533578 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:04.532587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:45223)
0809 13:10:04.533143 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:04.533303 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":306,"threads_started":1}
W20260809 13:10:04.534040 22756 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:04.543572 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:04.542398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:49255)
0809 13:10:04.543164 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:04.543332 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
W20260809 13:10:04.544049 22308 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:04.544680 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:04.543907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:36947)
0809 13:10:04.544421 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:04.544530 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":295,"threads_started":1}
W20260809 13:10:04.545023 22535 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.050853 22862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e05b48d43604a3483f7edd009414e60)
I20260809 13:10:05.051090 22862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } }
I20260809 13:10:05.051920 22863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e05b48d43604a3483f7edd009414e60)
I20260809 13:10:05.052147 22863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } }
I20260809 13:10:05.052542 22862 leader_election.cc:288] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275), 91251e300a394715b13800187f1ae370 (127.10.181.188:40825)
I20260809 13:10:05.053584 22863 leader_election.cc:288] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275), b07c6c2c1db443509128f0c57796c05e (127.10.181.189:32987)
I20260809 13:10:05.054416 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.052696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40960)
0809 13:10:05.053165 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.053365 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":236,"threads_started":1}
W20260809 13:10:05.054667 21739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.060784 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.053782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40964)
0809 13:10:05.054279 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.060534 (+ 6255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":250,"threads_started":1}
W20260809 13:10:05.066455 22028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.066697 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.066346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40976)
0809 13:10:05.066475 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.066557 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 13:10:05.066970 21739 leader_election.cc:334] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275): Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.068629 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.053376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:40825 (local address 127.0.0.1:51626)
0809 13:10:05.053878 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.053897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.054025 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.061233 (+ 7208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.061239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.061270 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.061739 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.061752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.063657 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.063664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.065954 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.065975 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.067755 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.067762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.067953 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.067993 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.068085 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.068235 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":2,"thread_start_us":260,"threads_started":1}
I20260809 13:10:05.068706 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.054320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.10.181.188:40825)
0809 13:10:05.060839 (+ 6519us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.060845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.060863 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:05.060930 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.060935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.061054 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.061908 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.062175 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.063471 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.066178 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.067484 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.067746 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.067827 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.068207 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.068333 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6383,"thread_start_us":6234,"threads_started":1}
I20260809 13:10:05.068797 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.068182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:40986)
0809 13:10:05.068389 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.068570 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 13:10:05.068950 22028 leader_election.cc:334] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275): Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.069697 22064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b07c6c2c1db443509128f0c57796c05e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "91251e300a394715b13800187f1ae370" is_pre_election: true
I20260809 13:10:05.070010 22064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b07c6c2c1db443509128f0c57796c05e in term 1.
I20260809 13:10:05.070551 21739 leader_election.cc:302] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91251e300a394715b13800187f1ae370, b07c6c2c1db443509128f0c57796c05e; no voters: 1e05b48d43604a3483f7edd009414e60
I20260809 13:10:05.070901 22862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 13:10:05.071040 22862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1e05b48d43604a3483f7edd009414e60)
I20260809 13:10:05.071158 22862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:10:05.072889 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.060961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.10.181.189:32987)
0809 13:10:05.066167 (+ 5206us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.066172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.066195 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:10:05.066252 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.066256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.066322 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.067059 (+ 737us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.067540 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.068795 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.070956 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.071931 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.072066 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.072124 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.072478 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.072583 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5071,"thread_start_us":4856,"threads_started":1}
I20260809 13:10:05.072885 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.060848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32987 (local address 127.0.0.1:52530)
0809 13:10:05.060979 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.060996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.061141 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.066483 (+ 5342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.066488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.066544 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.066904 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.066916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.068990 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.068996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.070731 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.070746 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.072108 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.072115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.072289 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.072322 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.072452 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.072604 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"thread_start_us":5300,"threads_started":1}
I20260809 13:10:05.073827 21780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "91251e300a394715b13800187f1ae370" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b07c6c2c1db443509128f0c57796c05e" is_pre_election: true
I20260809 13:10:05.074146 22862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } }
I20260809 13:10:05.074542 21780 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91251e300a394715b13800187f1ae370 in current term 2: Already voted for candidate b07c6c2c1db443509128f0c57796c05e in this term.
I20260809 13:10:05.074769 22862 leader_election.cc:288] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 election: Requested vote from peers 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275), 91251e300a394715b13800187f1ae370 (127.10.181.188:40825)
I20260809 13:10:05.075196 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.074882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41002)
0809 13:10:05.075013 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.075082 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 13:10:05.075155 22028 leader_election.cc:302] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91251e300a394715b13800187f1ae370; no voters: 1e05b48d43604a3483f7edd009414e60, b07c6c2c1db443509128f0c57796c05e
I20260809 13:10:05.075438 22863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:10:05.075947 22064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b07c6c2c1db443509128f0c57796c05e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "91251e300a394715b13800187f1ae370"
I20260809 13:10:05.076164 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.075783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41010)
0809 13:10:05.075939 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.076021 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 13:10:05.076326 21739 leader_election.cc:334] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275): Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
I20260809 13:10:05.077843 22863 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 13:10:05.080006 22064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b07c6c2c1db443509128f0c57796c05e in term 2.
I20260809 13:10:05.081015 21739 leader_election.cc:302] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91251e300a394715b13800187f1ae370, b07c6c2c1db443509128f0c57796c05e; no voters: 1e05b48d43604a3483f7edd009414e60
I20260809 13:10:05.081270 22862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:10:05.081717 22862 raft_consensus.cc:686] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [term 2 LEADER]: Becoming Leader. State: Replica: b07c6c2c1db443509128f0c57796c05e, State: Running, Role: LEADER
I20260809 13:10:05.082079 22862 consensus_queue.cc:237] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } }
I20260809 13:10:05.083652 22874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b07c6c2c1db443509128f0c57796c05e. Latest consensus state: current_term: 2 leader_uuid: "b07c6c2c1db443509128f0c57796c05e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:05.084031 22874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:05.084730 22877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:05.088750 22877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:05.089598 22877 catalog_manager.cc:1259] Loaded cluster ID: e6b64dcf847343ecae0b2ad70a3a59cd
I20260809 13:10:05.089679 22877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:05.090821 22877 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:05.091634 22877 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Loaded TSK: 0
I20260809 13:10:05.092490 22877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:10:05.110630 21762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:10:05.117673 21762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:05.122195 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.121813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41034)
0809 13:10:05.121986 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.122075 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:10:05.122195 22064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [term 2 FOLLOWER]: Refusing update from remote peer b07c6c2c1db443509128f0c57796c05e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 13:10:05.122908 22862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 13:10:05.123409 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.122982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41038)
0809 13:10:05.123158 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.123243 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:10:05.123626 21739 consensus_peers.cc:487] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e -> Peer 1e05b48d43604a3483f7edd009414e60 (127.10.181.190:46275): Couldn't send request to peer 1e05b48d43604a3483f7edd009414e60. Status: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 13:10:05.125708 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b07c6c2c1db443509128f0c57796c05e. Latest consensus state: current_term: 2 leader_uuid: "b07c6c2c1db443509128f0c57796c05e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:05.125984 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:05.127357 22874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b07c6c2c1db443509128f0c57796c05e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:05.127611 22874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:05.128065 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b07c6c2c1db443509128f0c57796c05e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1e05b48d43604a3483f7edd009414e60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46275 } } peers { permanent_uuid: "b07c6c2c1db443509128f0c57796c05e" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32987 } attrs { promote: false } } peers { permanent_uuid: "91251e300a394715b13800187f1ae370" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 40825 } attrs { promote: false } } }
I20260809 13:10:05.128350 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91251e300a394715b13800187f1ae370 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:10:05.142584 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.135294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:32775 (local address 127.0.0.1:39034)
0809 13:10:05.135455 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.135473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.135558 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.136269 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.136273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.136301 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.136525 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.136540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.138581 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.138589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.140506 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.140522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.141845 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.141851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.142033 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.142070 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.142150 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.142278 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":4}
I20260809 13:10:05.143585 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.135446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.10.181.129:32775)
0809 13:10:05.135925 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.135932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.135954 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:10:05.136027 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.136032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.136108 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.136217 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.136731 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.138406 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.140683 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.141582 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.142964 (+ 1382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.143041 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.143094 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.143234 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":445,"threads_started":1}
I20260809 13:10:05.145123 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.136700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33867 (local address 127.0.0.1:46844)
0809 13:10:05.136867 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.136884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.137008 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.138317 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.138322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.138351 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.138754 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.138767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.140305 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.140311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.142349 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.142363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.144436 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.144442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.144660 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.144692 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.144790 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.144900 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":2}
I20260809 13:10:05.145159 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa (DEFAULT_TABLE table=test-table [id=6f23709dcdf3465890ab8ca17b694d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:05.145474 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.136936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33365 (local address 127.0.0.1:34404)
0809 13:10:05.137566 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.137592 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.137702 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.138054 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.138059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.138082 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.138453 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.138473 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.139779 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.139783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.142882 (+ 3099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.142897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.144608 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.144613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.144728 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.144752 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.144843 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.144963 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":1,"thread_start_us":338,"threads_started":1}
I20260809 13:10:05.145666 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf6f4180cba4a739fcf919fb525e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:05.145666 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.137330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.10.181.130:33365)
0809 13:10:05.137796 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.137802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.137819 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:10:05.137876 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.137879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.137935 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.138017 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.138689 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.139639 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.143058 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.144259 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.144395 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.144461 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.145206 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.145347 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":241,"threads_started":1}
I20260809 13:10:05.146194 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.137416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.10.181.131:33867)
0809 13:10:05.137981 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.137987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.138006 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:10:05.138083 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.138088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.138168 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.138904 (+ 736us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.139094 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.140156 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.142541 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.144145 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.144346 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.144416 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.145689 (+ 1273us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.145896 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":318,"threads_started":1}
I20260809 13:10:05.146404 22436 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa (DEFAULT_TABLE table=test-table [id=6f23709dcdf3465890ab8ca17b694d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:05.146849 22436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf6f4180cba4a739fcf919fb525e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:05.146945 22653 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa (DEFAULT_TABLE table=test-table [id=6f23709dcdf3465890ab8ca17b694d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:05.147347 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf6f4180cba4a739fcf919fb525e7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:05.152151 22893 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Bootstrap starting.
I20260809 13:10:05.153890 22893 tablet_bootstrap.cc:654] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:05.157483 22894 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Bootstrap starting.
I20260809 13:10:05.158870 22894 tablet_bootstrap.cc:654] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:05.160046 22898 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Bootstrap starting.
I20260809 13:10:05.161368 22893 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: No bootstrap required, opened a new log
I20260809 13:10:05.161528 22893 ts_tablet_manager.cc:1397] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260809 13:10:05.162065 22898 tablet_bootstrap.cc:654] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:05.162196 22893 raft_consensus.cc:348] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.162396 22893 raft_consensus.cc:374] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:05.162484 22893 raft_consensus.cc:729] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92694ab93e74051888e08196b620fcb, State: Initialized, Role: FOLLOWER
I20260809 13:10:05.162727 22893 consensus_queue.cc:260] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.164208 22898 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: No bootstrap required, opened a new log
I20260809 13:10:05.164345 22898 ts_tablet_manager.cc:1397] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 13:10:05.164973 22898 raft_consensus.cc:348] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.165162 22898 raft_consensus.cc:374] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:05.165259 22898 raft_consensus.cc:729] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899d077a68fb4e1882ebcbfe1e78f502, State: Initialized, Role: FOLLOWER
I20260809 13:10:05.165529 22898 consensus_queue.cc:260] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.165967 22893 ts_tablet_manager.cc:1428] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 13:10:05.166287 22894 tablet_bootstrap.cc:492] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: No bootstrap required, opened a new log
I20260809 13:10:05.166462 22894 ts_tablet_manager.cc:1397] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 13:10:05.167421 22894 raft_consensus.cc:348] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.167771 22894 raft_consensus.cc:374] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:05.167887 22894 raft_consensus.cc:729] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1051ac8c4824d28b26a7851e08efa4d, State: Initialized, Role: FOLLOWER
I20260809 13:10:05.168129 22894 consensus_queue.cc:260] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.168967 22894 ts_tablet_manager.cc:1428] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 13:10:05.171729 22314 heartbeater.cc:503] Master 127.10.181.189:32987 was elected leader, sending a full tablet report...
I20260809 13:10:05.174293 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.166278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:47095)
0809 13:10:05.171109 (+ 4831us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.173984 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":958,"threads_started":1}
I20260809 13:10:05.174978 22898 ts_tablet_manager.cc:1428] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260809 13:10:05.176487 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.175544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:52717)
0809 13:10:05.176034 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.176161 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":263,"threads_started":1}
I20260809 13:10:05.176657 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.174838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:45213)
0809 13:10:05.176423 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.176510 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1437}
I20260809 13:10:05.177960 22537 heartbeater.cc:503] Master 127.10.181.189:32987 was elected leader, sending a full tablet report...
I20260809 13:10:05.178339 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.177031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:49689)
0809 13:10:05.177691 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.177952 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":430,"threads_started":1}
I20260809 13:10:05.182334 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.169204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:53359)
0809 13:10:05.181919 (+ 12715us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.182084 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12593,"thread_start_us":12440,"threads_started":1}
I20260809 13:10:05.183404 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.183030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:56309)
0809 13:10:05.183183 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.183267 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 13:10:05.183732 22759 heartbeater.cc:503] Master 127.10.181.189:32987 was elected leader, sending a full tablet report...
I20260809 13:10:05.464519 22907 raft_consensus.cc:482] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:05.464769 22907 raft_consensus.cc:504] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.466493 22907 leader_election.cc:288] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a92694ab93e74051888e08196b620fcb (127.10.181.129:32775), b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:10:05.472764 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.466950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:39011 (local address 127.10.181.131:33867)
0809 13:10:05.467156 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.467162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.467476 (+ 314us) server_negotiation.cc:407] Connection header received
0809 13:10:05.467642 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.467645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.467707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.467794 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.468332 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.469043 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.470985 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.471821 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.472003 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.472084 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.472364 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.472494 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 13:10:05.472862 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.467036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:33867 (local address 127.10.181.130:39011)
0809 13:10:05.467347 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.467361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.467513 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.467831 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.467836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.467899 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.468160 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.468169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.469192 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.469196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.470747 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.470760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.472020 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.472026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.472194 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.472218 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.472309 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.472447 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":36,"spinlock_wait_cycles":3968}
I20260809 13:10:05.472806 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.466919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:56273 (local address 127.10.181.129:32775)
0809 13:10:05.467170 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.467176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.467358 (+ 182us) server_negotiation.cc:407] Connection header received
0809 13:10:05.467604 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.467610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.467681 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.467748 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:05.468305 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.469020 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.471020 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.471873 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.472046 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.472131 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.472426 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.472554 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 13:10:05.473033 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.466952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:32775 (local address 127.10.181.130:56273)
0809 13:10:05.467226 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.467245 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:05.467392 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:05.467797 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:05.467802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:05.467828 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:05.468149 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.468159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.469164 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.469169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.470742 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:05.470757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.472129 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.472136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:05.472257 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.472279 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 13:10:05.472371 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:10:05.472513 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":40}
I20260809 13:10:05.473724 22685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf6f4180cba4a739fcf919fb525e7aa" candidate_uuid: "899d077a68fb4e1882ebcbfe1e78f502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1051ac8c4824d28b26a7851e08efa4d" is_pre_election: true
I20260809 13:10:05.473963 22685 raft_consensus.cc:2454] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899d077a68fb4e1882ebcbfe1e78f502 in term 0.
I20260809 13:10:05.473879 22248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf6f4180cba4a739fcf919fb525e7aa" candidate_uuid: "899d077a68fb4e1882ebcbfe1e78f502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92694ab93e74051888e08196b620fcb" is_pre_election: true
I20260809 13:10:05.474319 22248 raft_consensus.cc:2454] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 899d077a68fb4e1882ebcbfe1e78f502 in term 0.
I20260809 13:10:05.474524 22363 leader_election.cc:302] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899d077a68fb4e1882ebcbfe1e78f502, b1051ac8c4824d28b26a7851e08efa4d; no voters:
I20260809 13:10:05.474975 22907 raft_consensus.cc:2781] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:05.475091 22907 raft_consensus.cc:482] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:05.475194 22907 raft_consensus.cc:3037] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:05.478235 22907 raft_consensus.cc:504] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.478879 22907 leader_election.cc:288] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [CANDIDATE]: Term 1 election: Requested vote from peers a92694ab93e74051888e08196b620fcb (127.10.181.129:32775), b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867)
I20260809 13:10:05.479365 22248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf6f4180cba4a739fcf919fb525e7aa" candidate_uuid: "899d077a68fb4e1882ebcbfe1e78f502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a92694ab93e74051888e08196b620fcb"
I20260809 13:10:05.479475 22685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf6f4180cba4a739fcf919fb525e7aa" candidate_uuid: "899d077a68fb4e1882ebcbfe1e78f502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1051ac8c4824d28b26a7851e08efa4d"
I20260809 13:10:05.479596 22248 raft_consensus.cc:3037] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:05.479677 22685 raft_consensus.cc:3037] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:05.482616 22685 raft_consensus.cc:2454] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899d077a68fb4e1882ebcbfe1e78f502 in term 1.
I20260809 13:10:05.482627 22248 raft_consensus.cc:2454] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 899d077a68fb4e1882ebcbfe1e78f502 in term 1.
I20260809 13:10:05.483024 22363 leader_election.cc:302] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 899d077a68fb4e1882ebcbfe1e78f502, b1051ac8c4824d28b26a7851e08efa4d; no voters:
I20260809 13:10:05.483291 22907 raft_consensus.cc:2781] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:05.483765 22907 raft_consensus.cc:686] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 LEADER]: Becoming Leader. State: Replica: 899d077a68fb4e1882ebcbfe1e78f502, State: Running, Role: LEADER
I20260809 13:10:05.484187 22907 consensus_queue.cc:237] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } }
I20260809 13:10:05.484740 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.484320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:41947)
0809 13:10:05.484495 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.484606 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:10:05.487646 21763 catalog_manager.cc:5495] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 reported cstate change: term changed from 0 to 1, leader changed from <none> to 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "899d077a68fb4e1882ebcbfe1e78f502" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "899d077a68fb4e1882ebcbfe1e78f502" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33365 } health_report { overall_health: HEALTHY } } }
I20260809 13:10:05.508790 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.503884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.10.181.130:33365)
0809 13:10:05.504066 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.504080 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.504097 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:10:05.504281 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.504284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.504364 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.504487 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:05.504798 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.505542 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.506855 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.507325 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.507434 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.507833 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:05.508362 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:05.508472 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.508514 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.508586 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:10:05.512868 22685 raft_consensus.cc:1261] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Refusing update from remote peer 899d077a68fb4e1882ebcbfe1e78f502: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:05.513166 22248 raft_consensus.cc:1261] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Refusing update from remote peer 899d077a68fb4e1882ebcbfe1e78f502: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:05.513664 22913 consensus_queue.cc:1035] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1051ac8c4824d28b26a7851e08efa4d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:05.514142 22907 consensus_queue.cc:1035] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a92694ab93e74051888e08196b620fcb" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:05.517141 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.516475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:58393)
0809 13:10:05.516759 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.516905 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 13:10:05.518330 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.517824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:45969)
0809 13:10:05.518025 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.518148 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:10:05.524446 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.523890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.129:50665)
0809 13:10:05.524168 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.524258 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 13:10:05.526648 22922 mvcc.cc:205] Tried to move back new op lower bound from 7316606998574055424 to 7316606998467174400. Current Snapshot: MvccSnapshot[applied={T|T < 7316606998574055424 or (T in {7316606998574055424})}]
I20260809 13:10:05.528366 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.527775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.130:46375)
0809 13:10:05.528099 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.528202 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260809 13:10:05.533763 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.527032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.10.181.131:51873)
0809 13:10:05.533558 (+ 6526us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.533636 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6411}
I20260809 13:10:05.537940 22921 mvcc.cc:205] Tried to move back new op lower bound from 7316606998574055424 to 7316606998467174400. Current Snapshot: MvccSnapshot[applied={T|T < 7316606998574055424}]
I20260809 13:10:05.540477 21762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:05.540783 21762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:05.546371 21762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b07c6c2c1db443509128f0c57796c05e: Sending DeleteTablet for 3 replicas of tablet 2cf6f4180cba4a739fcf919fb525e7aa
I20260809 13:10:05.547420 22436 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:05 UTC) from {username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:34404
I20260809 13:10:05.547670 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:05 UTC) from {username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:46844
I20260809 13:10:05.547895 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf6f4180cba4a739fcf919fb525e7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:05 UTC) from {username='oryx', principal='oryx/127.10.181.189@KRBTEST.COM'} at 127.0.0.1:39034
I20260809 13:10:05.549739 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 22032
I20260809 13:10:05.556345 22933 tablet_replica.cc:330] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: stopping tablet replica
I20260809 13:10:05.556551 22936 tablet_replica.cc:330] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: stopping tablet replica
I20260809 13:10:05.556562 22933 raft_consensus.cc:2229] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:05.556742 22936 raft_consensus.cc:2229] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:05.557058 22936 raft_consensus.cc:2258] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:05.557166 22933 raft_consensus.cc:2258] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:05.558790 22936 ts_tablet_manager.cc:1905] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:05.559077 22933 ts_tablet_manager.cc:1905] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:05.566250 22936 ts_tablet_manager.cc:1918] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:05.566458 22936 log.cc:1196] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/2cf6f4180cba4a739fcf919fb525e7aa
I20260809 13:10:05.566982 22936 ts_tablet_manager.cc:1939] T 2cf6f4180cba4a739fcf919fb525e7aa P a92694ab93e74051888e08196b620fcb: Deleting consensus metadata
I20260809 13:10:05.568343 21740 catalog_manager.cc:4841] TS a92694ab93e74051888e08196b620fcb (127.10.181.129:32775): tablet 2cf6f4180cba4a739fcf919fb525e7aa (table test-table [id=6f23709dcdf3465890ab8ca17b694d7f]) successfully deleted
I20260809 13:10:05.572055 22935 tablet_replica.cc:330] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: stopping tablet replica
I20260809 13:10:05.572254 22935 raft_consensus.cc:2229] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:05.572562 22935 raft_consensus.cc:2258] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:05.577143 22935 ts_tablet_manager.cc:1905] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:05.583601 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.550846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.10.181.129:32775)
0809 13:10:05.551022 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.551028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.557864 (+ 6836us) server_negotiation.cc:407] Connection header received
0809 13:10:05.565595 (+ 7731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.565603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.565702 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.565791 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:05.568376 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.569587 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.571589 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.572235 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.572374 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.576320 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:05.576339 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:05.576361 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:05.576399 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:05.580234 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.581085 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.581096 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.581100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.581251 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.581738 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.581743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.581747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.582202 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:05.582461 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.583024 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.583253 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:10:05.585011 22301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:05.585003 22935 ts_tablet_manager.cc:1918] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:05.585129 22935 log.cc:1196] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/2cf6f4180cba4a739fcf919fb525e7aa
I20260809 13:10:05.585600 22935 ts_tablet_manager.cc:1939] T 2cf6f4180cba4a739fcf919fb525e7aa P b1051ac8c4824d28b26a7851e08efa4d: Deleting consensus metadata
I20260809 13:10:05.586709 21741 catalog_manager.cc:4841] TS b1051ac8c4824d28b26a7851e08efa4d (127.10.181.131:33867): tablet 2cf6f4180cba4a739fcf919fb525e7aa (table test-table [id=6f23709dcdf3465890ab8ca17b694d7f]) successfully deleted
I20260809 13:10:05.587881 22933 ts_tablet_manager.cc:1918] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:05.588032 22933 log.cc:1196] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/2cf6f4180cba4a739fcf919fb525e7aa
I20260809 13:10:05.588495 22933 ts_tablet_manager.cc:1939] T 2cf6f4180cba4a739fcf919fb525e7aa P 899d077a68fb4e1882ebcbfe1e78f502: Deleting consensus metadata
I20260809 13:10:05.589614 21741 catalog_manager.cc:4841] TS 899d077a68fb4e1882ebcbfe1e78f502 (127.10.181.130:33365): tablet 2cf6f4180cba4a739fcf919fb525e7aa (table test-table [id=6f23709dcdf3465890ab8ca17b694d7f]) successfully deleted
I20260809 13:10:05.606685 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.606205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41048)
0809 13:10:05.606395 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.606521 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:10:05.607856 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.607445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46275 (local address 127.0.0.1:41058)
0809 13:10:05.607603 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:05.607710 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 13:10:05.713761 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 22032
I20260809 13:10:05.747390 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 22331
I20260809 13:10:05.761461 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.748231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.10.181.130:33365)
0809 13:10:05.748447 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.748453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.748529 (+ 76us) server_negotiation.cc:407] Connection header received
0809 13:10:05.748741 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.748746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.748856 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.748946 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:05.750350 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.751177 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.752411 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.752840 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.752976 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.755494 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:05.755506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:05.755525 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:05.755569 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:05.758590 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.759261 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.759268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.759271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.759474 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.759877 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.759881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.759883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.760265 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:05.760589 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.760901 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.761129 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 13:10:05.762157 22524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:05.881433 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 22331
I20260809 13:10:05.909228 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 22559
I20260809 13:10:05.922477 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:05.910062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.10.181.131:33867)
0809 13:10:05.910253 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:05.910258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:05.910277 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:10:05.910431 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:05.910435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:05.910501 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:05.910610 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:05.911849 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.912565 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.913871 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:05.914278 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:05.914375 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:05.916705 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:05.916720 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:05.916738 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:05.916769 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:05.919686 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.920359 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.920368 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.920370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.920495 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:05.920871 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:05.920874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:05.920876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:05.921234 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:05.921578 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:05.921971 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 13:10:05.922201 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:10:05.923094 22748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:06.016438 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 22559
I20260809 13:10:06.041288 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 21545
I20260809 13:10:06.042413 21807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:06.167964 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 21545
I20260809 13:10:06.197539 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 21695
I20260809 13:10:06.198694 22093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:06.316233 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 21695
2026-08-09T13:10:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10092 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:10:06 dist-test-slave-42z9 krb5kdc[22950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:10:06 dist-test-slave-42z9 krb5kdc[22950](info): set up 2 sockets
Aug 09 13:10:06 dist-test-slave-42z9 krb5kdc[22950](info): commencing operation
krb5kdc: starting...
W20260809 13:10:07.538349 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:10:07 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:10:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab.
Entry for principal oryx/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab.
I20260809 13:10:07.809096 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36861
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab
--principal=oryx/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:39425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:36861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:08.141901 22968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:08.142204 22968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:08.142340 22968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:08.149425 22968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:08.149552 22968 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 13:10:08.149616 22968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:08.149701 22968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:08.149770 22968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:08.149857 22968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:08.160791 22968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:39425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:36861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.190.keytab
--principal=oryx/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:08.162184 22968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:08.163501 22968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:08.174888 22978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:08.175011 22979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:08.175063 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:08.176074 22968 server_base.cc:1034] running on GCE node
I20260809 13:10:08.176805 22968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:08.178236 22968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:08.179469 22968 hybrid_clock.cc:648] HybridClock initialized: now 1786281008179408 us; error 31 us; skew 500 ppm
I20260809 13:10:08.179793 22968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:08 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281008, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:08.182926 22968 init.cc:377] Logged in from keytab as oryx/127.10.181.190@KRBTEST.COM (short username oryx)
I20260809 13:10:08.184373 22968 webserver.cc:460] Webserver started at http://127.10.181.190:44441/ using document root <none> and password file <none>
I20260809 13:10:08.185110 22968 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:08.185199 22968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:08.185580 22968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:08.188540 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "3b223145cfec4244bac9de369b4b381a"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "b5e50b10fd6553c9c2c0e782e7a01fd9"
server_key_iv: "b9cb43467a41bfab64a4bdbdcd6f38a1"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.189319 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "3b223145cfec4244bac9de369b4b381a"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "b5e50b10fd6553c9c2c0e782e7a01fd9"
server_key_iv: "b9cb43467a41bfab64a4bdbdcd6f38a1"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.194375 22968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:10:08.197906 22998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:08.199553 22968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:08.199728 22968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "3b223145cfec4244bac9de369b4b381a"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "b5e50b10fd6553c9c2c0e782e7a01fd9"
server_key_iv: "b9cb43467a41bfab64a4bdbdcd6f38a1"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.199934 22968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:08.216606 22968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:08.220733 22968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:08.220983 22968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:08.241706 22968 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:36861
I20260809 13:10:08.241727 23096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:36861 every 8 connection(s)
I20260809 13:10:08.243142 22968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:08.243911 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 22968
I20260809 13:10:08.244119 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:10:08.244853 10966 external_mini_cluster.cc:1428] Setting key 9fcf213ad74f79e3e8eacda8cd8a35f3
I20260809 13:10:08.247555 23098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:10:08 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:08.260357 23098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Bootstrap starting.
I20260809 13:10:08.263590 23098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:08.264900 23098 log.cc:824] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:08.267256 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:08.246781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.10.181.190:36861)
0809 13:10:08.247535 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:08.247550 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:08.247593 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:10:08.249258 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:08.249279 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:08.249684 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:08.249994 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:08.251759 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.253109 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.254865 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.255519 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.255640 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:08.260204 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:08.260213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:08.260227 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:08.260263 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:08.263164 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.263834 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.263843 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.263851 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.264087 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.264443 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.264448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.264451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.265184 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:08.265536 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:08.265909 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 13:10:08.266283 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":376,"threads_started":1}
I20260809 13:10:08.267930 23098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: No bootstrap required, opened a new log
I20260809 13:10:08.275460 23098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } }
I20260809 13:10:08.275770 23098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:08.275866 23098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b223145cfec4244bac9de369b4b381a, State: Initialized, Role: FOLLOWER
I20260809 13:10:08.276396 23098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } }
I20260809 13:10:08.276611 23098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:08.276738 23098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:08.276895 23098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:08.279127 23098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } }
I20260809 13:10:08.279662 23098 leader_election.cc:302] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b223145cfec4244bac9de369b4b381a; no voters:
I20260809 13:10:08.280196 23098 leader_election.cc:288] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:10:08.280303 23108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:08.280990 23108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [term 1 LEADER]: Becoming Leader. State: Replica: 3b223145cfec4244bac9de369b4b381a, State: Running, Role: LEADER
I20260809 13:10:08.281571 23108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } }
I20260809 13:10:08.282256 23098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:08.284723 23111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b223145cfec4244bac9de369b4b381a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } } }
I20260809 13:10:08.284772 23112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b223145cfec4244bac9de369b4b381a. Latest consensus state: current_term: 1 leader_uuid: "3b223145cfec4244bac9de369b4b381a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b223145cfec4244bac9de369b4b381a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36861 } } }
I20260809 13:10:08.285097 23111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:08.285187 23112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:08.286777 23117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:08.292053 23117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:08.301399 23117 catalog_manager.cc:1347] Generated new cluster ID: 3548933f628b482986b608794f26f83e
I20260809 13:10:08.301530 23117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:08.319564 23117 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:10:08.321112 23117 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:08.329978 23117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Generated new TSK 0
I20260809 13:10:08.331012 23117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
I20260809 13:10:08.405772 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:36861
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:39425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:08.740762 23141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:08.741039 23141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:08.741189 23141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:08.748287 23141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:08.748407 23141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:08.748539 23141 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:10:08.748660 23141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:08.758208 23141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:39425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:08.759490 23141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:08.760835 23141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:08.773542 23147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:08.773468 23152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:08.773663 23149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:08.774746 23141 server_base.cc:1034] running on GCE node
I20260809 13:10:08.775334 23141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:08.776209 23141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:08.777424 23141 hybrid_clock.cc:648] HybridClock initialized: now 1786281008777353 us; error 49 us; skew 500 ppm
I20260809 13:10:08.777743 23141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:08 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281008, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:08.780781 23141 init.cc:377] Logged in from keytab as oryx/127.10.181.129@KRBTEST.COM (short username oryx)
I20260809 13:10:08.782254 23141 webserver.cc:460] Webserver started at http://127.10.181.129:46473/ using document root <none> and password file <none>
I20260809 13:10:08.782951 23141 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:08.783061 23141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:08.783371 23141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:08.786252 23141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "fbe09710ae154273a301d33cef7ca812"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "43f49193e7ab7381235ce6348936b44f"
server_key_iv: "839c0a1c7845c7fa21efeb256a046d03"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.787094 23141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "fbe09710ae154273a301d33cef7ca812"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "43f49193e7ab7381235ce6348936b44f"
server_key_iv: "839c0a1c7845c7fa21efeb256a046d03"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.792099 23141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:10:08.795466 23164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:08.796734 23141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:08.796880 23141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "fbe09710ae154273a301d33cef7ca812"
format_stamp: "Formatted at 2026-08-09 13:10:08 on dist-test-slave-42z9"
server_key: "43f49193e7ab7381235ce6348936b44f"
server_key_iv: "839c0a1c7845c7fa21efeb256a046d03"
server_key_version: "encryptionkey@0"
I20260809 13:10:08.797086 23141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:08.820204 23141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:08.824191 23141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:08.824430 23141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:08.827101 23141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:08.827186 23141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:08.827353 23141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:08.827448 23141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:10:08 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281008, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:08.847699 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:08.827502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:51955 (local address 127.10.181.190:36861)
0809 13:10:08.828335 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:08.828341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:08.829315 (+ 974us) server_negotiation.cc:407] Connection header received
0809 13:10:08.831927 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:08.831933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:08.832029 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:08.832156 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:08.834533 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.835450 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.837031 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.837603 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.837735 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:08.841930 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:08.841936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:08.841940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:08.841973 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:08.844295 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.844998 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.845003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.845007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.845131 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.845603 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.845609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.845612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.845927 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:08.846102 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:08.847106 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0809 13:10:08.847338 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":595,"threads_started":1}
I20260809 13:10:08.849303 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:08.828076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.129:51955)
0809 13:10:08.828992 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:08.829050 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:08.831589 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:08.832432 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:08.832444 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:08.833289 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:08.834298 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:08.834331 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.835610 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.835614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:08.836880 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:08.836893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.837873 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.837881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:08.838201 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:08.838970 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:08.838995 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:08.841710 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:08.844441 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:08.844450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:08.844465 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:08.844863 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:08.845242 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:08.845247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:08.845251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:08.845481 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:08.846090 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:08.846099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:08.846581 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 13:10:08.846981 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 13:10:08.847581 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":351,"threads_started":1}
I20260809 13:10:08.872561 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:08.856638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:46525 (local address 127.10.181.190:36861)
0809 13:10:08.856863 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:08.856869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:08.858275 (+ 1406us) server_negotiation.cc:407] Connection header received
0809 13:10:08.858446 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:08.858452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:08.858541 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:08.858645 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:08.859723 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.860807 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.862967 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.863620 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.863748 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:08.865996 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:08.866005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:08.866010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:08.866052 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:08.869470 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.870355 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.870361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.870365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.870532 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:08.870900 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:08.870906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:08.870908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:08.871301 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:08.871465 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:08.871965 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 13:10:08.872213 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:10:08.872762 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:08.856618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.129:46525)
0809 13:10:08.858136 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:08.858160 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:08.858308 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:08.858696 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:08.858701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:08.859268 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:08.859572 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:08.859582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.860975 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.860980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:08.862794 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:08.862813 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:08.863842 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:08.863850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:08.864041 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:08.864708 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:08.864724 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:08.865784 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:08.869606 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:08.869611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:08.869615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:08.870187 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:08.870648 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:08.870651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:08.870653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:08.870773 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:08.871476 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:08.871480 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:08.871681 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 13:10:08.871828 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 13:10:08.872175 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":1246,"threads_started":1}
I20260809 13:10:08.872957 23141 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:44505
I20260809 13:10:08.872982 23366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:44505 every 8 connection(s)
I20260809 13:10:08.874384 23141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:08.876719 23367 heartbeater.cc:347] Connected to a master server at 127.10.181.190:36861
I20260809 13:10:08.876928 23367 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:08.877576 23367 heartbeater.cc:511] Master 127.10.181.190:36861 requested a full tablet report, sending...
I20260809 13:10:08.879446 23024 ts_manager.cc:194] Registered new tserver with Master: fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505)
I20260809 13:10:08.880836 23024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:51955
I20260809 13:10:08.881850 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 23141
I20260809 13:10:08.882074 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:10:08.882915 10966 external_mini_cluster.cc:1428] Setting key 69debbb9cd8159ab0976cc1ea31c9e65
WARNING: no policy specified for oryx/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
I20260809 13:10:08.936669 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:36861
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:39425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:09.274475 23373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:09.274753 23373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:09.274858 23373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:09.282147 23373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:09.282256 23373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:09.282346 23373 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:10:09.282433 23373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:09.292194 23373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:39425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:09.293494 23373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:09.294924 23373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:09.308077 23380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:09.308013 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:09.308046 23387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:09.308782 23373 server_base.cc:1034] running on GCE node
I20260809 13:10:09.309360 23373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:09.310372 23373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:09.311594 23373 hybrid_clock.cc:648] HybridClock initialized: now 1786281009311518 us; error 44 us; skew 500 ppm
I20260809 13:10:09.311888 23373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:09 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281009, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:09.314978 23373 init.cc:377] Logged in from keytab as oryx/127.10.181.130@KRBTEST.COM (short username oryx)
I20260809 13:10:09.316385 23373 webserver.cc:460] Webserver started at http://127.10.181.130:40037/ using document root <none> and password file <none>
I20260809 13:10:09.317051 23373 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:09.317147 23373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:09.317490 23373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:09.320333 23373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "bd6b48667e68493090df6dad37603702"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "87a9f42900e67f74c8f226b24cc44e0b"
server_key_iv: "deea2686e089ca20caf9f18c54cf3418"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.321089 23373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "bd6b48667e68493090df6dad37603702"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "87a9f42900e67f74c8f226b24cc44e0b"
server_key_iv: "deea2686e089ca20caf9f18c54cf3418"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.325943 23373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:10:09.329226 23400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:09.330531 23373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:09.330698 23373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "bd6b48667e68493090df6dad37603702"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "87a9f42900e67f74c8f226b24cc44e0b"
server_key_iv: "deea2686e089ca20caf9f18c54cf3418"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.330891 23373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:09.356516 23373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:09.360678 23373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:09.360930 23373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:09.363718 23373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:09.363830 23373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:09.364019 23373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:09.364097 23373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:10:09 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281009, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:09.385023 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.364068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:43289 (local address 127.10.181.190:36861)
0809 13:10:09.364269 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:09.364276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:09.365776 (+ 1500us) server_negotiation.cc:407] Connection header received
0809 13:10:09.367804 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:09.367810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:09.367905 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:09.368021 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:09.370069 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.371200 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.373772 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.374278 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.374420 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.378869 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:09.378877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:09.378882 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:09.378927 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:09.381945 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.382703 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.382708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.382711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.382833 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.383302 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.383306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.383308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.383584 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:09.383744 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:09.384450 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 13:10:09.384672 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:10:09.386242 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.364745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.130:43289)
0809 13:10:09.365607 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:09.365656 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:09.367463 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:09.368221 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:09.368228 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:09.368841 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:09.369854 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.369887 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.371400 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.371406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.373616 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.373630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.374534 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.374544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.374825 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.375587 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:09.375634 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:09.378572 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:09.382088 (+ 3516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.382101 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.382137 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.382560 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.382950 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.382955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.382959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.383162 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.383731 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:09.383738 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:09.384067 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 13:10:09.384295 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 13:10:09.384789 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
I20260809 13:10:09.405342 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.392468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59959 (local address 127.10.181.190:36861)
0809 13:10:09.392658 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:09.392662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:09.393417 (+ 755us) server_negotiation.cc:407] Connection header received
0809 13:10:09.393603 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:09.393608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:09.393676 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:09.393752 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:09.394668 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.395368 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.396662 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.397069 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.397177 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.399393 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:09.399399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:09.399402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:09.399433 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:09.402515 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.403423 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.403428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.403431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.403571 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.404022 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.404025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.404027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.404349 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:09.404461 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:09.404964 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 13:10:09.405113 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:10:09.405752 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.392466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.130:59959)
0809 13:10:09.393313 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:09.393331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:09.393474 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:09.393813 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:09.393818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:09.394236 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:09.394518 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.394527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.395503 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.395509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.396507 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.396519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.397263 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.397270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.397435 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.398141 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:09.398158 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:09.399122 (+ 964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:09.402677 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.402683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.402687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.403244 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.403711 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.403716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.403721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.403901 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.404479 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:09.404485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:09.404722 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 13:10:09.404888 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 13:10:09.405249 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":363,"threads_started":1}
I20260809 13:10:09.418721 23373 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:34281
I20260809 13:10:09.418752 23590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:34281 every 8 connection(s)
I20260809 13:10:09.419894 23373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:09.420315 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 23373
I20260809 13:10:09.420502 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:10:09.421077 10966 external_mini_cluster.cc:1428] Setting key ad83de032acc555ee2d80c9866ee6421
I20260809 13:10:09.422061 23591 heartbeater.cc:347] Connected to a master server at 127.10.181.190:36861
I20260809 13:10:09.422287 23591 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:09.422921 23591 heartbeater.cc:511] Master 127.10.181.190:36861 requested a full tablet report, sending...
I20260809 13:10:09.424264 23024 ts_manager.cc:194] Registered new tserver with Master: bd6b48667e68493090df6dad37603702 (127.10.181.130:34281)
I20260809 13:10:09.425326 23024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:43289
WARNING: no policy specified for oryx/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
I20260809 13:10:09.478579 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:36861
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:39425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:09.809178 23599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:09.809456 23599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:09.809602 23599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:09.816596 23599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:09.816702 23599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:09.816823 23599 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 13:10:09.816933 23599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:09.826586 23599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:39425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.181.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:09.827814 23599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:09.829125 23599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:09.842017 23606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:09.841985 23607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:09.842059 23611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:09.842764 23599 server_base.cc:1034] running on GCE node
I20260809 13:10:09.843346 23599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:09.844202 23599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:09.845484 23599 hybrid_clock.cc:648] HybridClock initialized: now 1786281009845369 us; error 37 us; skew 500 ppm
I20260809 13:10:09.845846 23599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:09 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281009, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:09.849005 23599 init.cc:377] Logged in from keytab as oryx/127.10.181.131@KRBTEST.COM (short username oryx)
I20260809 13:10:09.850477 23599 webserver.cc:460] Webserver started at http://127.10.181.131:45577/ using document root <none> and password file <none>
I20260809 13:10:09.851153 23599 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:09.851254 23599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:09.851562 23599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:09.854468 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "313647ff35bb4b67b54734487abd8958"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "bce9804f1f2d0f6141161a6356a04306"
server_key_iv: "dbee7e5a0d92f54cd5cba2d651cfa9b1"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.855201 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "313647ff35bb4b67b54734487abd8958"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "bce9804f1f2d0f6141161a6356a04306"
server_key_iv: "dbee7e5a0d92f54cd5cba2d651cfa9b1"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.860044 23599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:10:09.863394 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:09.864794 23599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:09.864938 23599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "313647ff35bb4b67b54734487abd8958"
format_stamp: "Formatted at 2026-08-09 13:10:09 on dist-test-slave-42z9"
server_key: "bce9804f1f2d0f6141161a6356a04306"
server_key_iv: "dbee7e5a0d92f54cd5cba2d651cfa9b1"
server_key_version: "encryptionkey@0"
I20260809 13:10:09.865162 23599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:09.883884 23367 heartbeater.cc:503] Master 127.10.181.190:36861 was elected leader, sending a full tablet report...
I20260809 13:10:09.902341 23599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:09.906625 23599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:09.906874 23599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:09.909430 23599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:09.909562 23599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:09.909693 23599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:09.909751 23599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 13:10:09 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281009, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:09.932035 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.910103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:56473 (local address 127.10.181.190:36861)
0809 13:10:09.910584 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:09.910591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:09.911806 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 13:10:09.914613 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:09.914628 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:09.914725 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:09.914852 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:09.917175 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.918077 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.920573 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.920989 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.921113 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.925728 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:09.925736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:09.925741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:09.925783 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:09.928824 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.929535 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.929541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.929545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.929693 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.930161 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.930168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.930171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.930543 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:09.930699 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:09.931424 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 13:10:09.931659 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260809 13:10:09.932940 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.910736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.131:56473)
0809 13:10:09.911615 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:09.911674 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:09.914294 (+ 2620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:09.915149 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:09.915163 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:09.916026 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:09.916949 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.916982 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.918256 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.918261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.920430 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.920443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.921254 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.921263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.921503 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.922245 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:09.922288 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:09.925463 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:09.928966 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.928975 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.928992 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.929397 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.929811 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.929816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.929820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.930028 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.930686 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:09.930694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:09.931040 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 13:10:09.931281 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 13:10:09.931772 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":351,"threads_started":1}
I20260809 13:10:09.952203 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.938486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.10.181.131:33391)
0809 13:10:09.939045 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:09.939061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:09.939185 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:09.939529 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:09.939534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:09.939958 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:09.940234 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.940243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.941204 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.941208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.942556 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:09.942572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.943791 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.943801 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:09.944032 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.944753 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:09.944790 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:09.945920 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:09.949483 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.949489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.949493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.950043 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.950433 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:09.950438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:09.950442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:09.950641 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:09.951316 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:09.951321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:09.951513 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 13:10:09.951651 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 13:10:09.951879 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":309,"threads_started":1}
I20260809 13:10:09.952436 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:09.938390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:33391 (local address 127.10.181.190:36861)
0809 13:10:09.938547 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:09.938551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:09.939158 (+ 607us) server_negotiation.cc:407] Connection header received
0809 13:10:09.939294 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:09.939299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:09.939368 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:09.939479 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:09.940379 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.941058 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.942760 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:09.943336 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:09.943410 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:09.946130 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:09.946137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:09.946142 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:09.946183 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:09.949331 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.950166 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.950170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.950172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.950302 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:09.950811 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:09.950817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:09.950821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:09.951187 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:09.951364 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:09.951815 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 13:10:09.952070 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:10:09.962409 23599 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:40159
I20260809 13:10:09.962440 23807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:40159 every 8 connection(s)
I20260809 13:10:09.963541 23599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:09.964087 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 23599
I20260809 13:10:09.964247 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:10:09.964834 10966 external_mini_cluster.cc:1428] Setting key 96c3aa653507254b6b3c30497c8a692c
I20260809 13:10:09.965597 23810 heartbeater.cc:347] Connected to a master server at 127.10.181.190:36861
I20260809 13:10:09.965847 23810 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:09.966511 23810 heartbeater.cc:511] Master 127.10.181.190:36861 requested a full tablet report, sending...
I20260809 13:10:09.967851 23024 ts_manager.cc:194] Registered new tserver with Master: 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:09.968680 23024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:56473
I20260809 13:10:09.979934 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 13:10:10 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281010, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 13:10:10 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:10 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 13:10:10.037788 23637 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:10.027518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.10.181.190:36861)
0809 13:10:10.027786 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.027792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.027979 (+ 187us) server_negotiation.cc:407] Connection header received
0809 13:10:10.028153 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.028157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.028233 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.028322 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:10.029602 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.030316 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.031467 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.031862 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.031951 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.037485 (+ 5534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41472: BlockingRecv error: recv got EOF from 127.0.0.1:41472 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 09 13:10:10 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281010, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:10.053472 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.040804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.10.181.190:36861)
0809 13:10:10.041030 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.041035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.041243 (+ 208us) server_negotiation.cc:407] Connection header received
0809 13:10:10.041371 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.041376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.041464 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.041556 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:10.042602 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.043282 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.044422 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.044835 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.044924 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.048473 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:10.048478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:10.048482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:10.048514 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:10.050909 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:10.051575 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:10.051579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:10.051582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:10.051708 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:10.052072 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:10.052076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:10.052078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:10.052387 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:10.052669 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.052948 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.053172 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:10:10.057422 23024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:10:10.064267 23024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:10.098953 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.077515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:34281 (local address 127.0.0.1:34666)
0809 13:10:10.090803 (+ 13288us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.090819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.090925 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.091684 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.091690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.091718 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.092073 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.092086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.093917 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.093921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.096027 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.096040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.098002 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.098009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.098218 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.098275 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.098405 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.098611 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13171,"thread_start_us":13039,"threads_started":1}
I20260809 13:10:10.098966 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.078049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.10.181.130:34281)
0809 13:10:10.090369 (+ 12320us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.090380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.091049 (+ 669us) server_negotiation.cc:407] Connection header received
0809 13:10:10.091194 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.091202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.091459 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.091668 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.092279 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.093755 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.096199 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.097505 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.097686 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.097870 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.098490 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.098657 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12170,"thread_start_us":12046,"threads_started":1}
I20260809 13:10:10.100387 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.076147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:44505 (local address 127.0.0.1:49938)
0809 13:10:10.082829 (+ 6682us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.082871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.083050 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.093255 (+ 10205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.093260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.093285 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.093720 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.093732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.095534 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.095543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.097779 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.097796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.099640 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.099647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.099819 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.099850 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.099927 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.100055 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":575,"threads_started":1}
I20260809 13:10:10.100876 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.076372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.10.181.129:44505)
0809 13:10:10.092673 (+ 16301us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.092679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.092702 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:10:10.092778 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.092783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.092980 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.093231 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.093922 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.095389 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.097974 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.099368 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.100243 (+ 875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.100382 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.100472 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.100632 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16149,"thread_start_us":16012,"threads_started":1}
I20260809 13:10:10.101235 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.077347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.10.181.131:40159)
0809 13:10:10.092383 (+ 15036us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.092390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.092422 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:10:10.092512 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.092518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.092730 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.092905 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.093718 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.095133 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.097894 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.099368 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.099602 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.099775 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.100728 (+ 953us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.100896 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14854,"thread_start_us":14665,"threads_started":1}
I20260809 13:10:10.102062 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.076834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40159 (local address 127.0.0.1:57702)
0809 13:10:10.089606 (+ 12772us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.089625 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.089735 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.092936 (+ 3201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.092941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.092966 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.093429 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.093471 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.095274 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.095279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.097532 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.097549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.099925 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.099942 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.100091 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.100121 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.100205 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.101668 (+ 1463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12621,"mutex_wait_us":22,"thread_start_us":12934,"threads_started":1}
I20260809 13:10:10.103365 23473 tablet_service.cc:1467] Processing CreateTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 (DEFAULT_TABLE table=test-table [id=b01801e0c99641f6be4714e9ebd8689f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.104769 23473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ac5f35cd8a464d8333c42b911eb216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.104511 23257 tablet_service.cc:1467] Processing CreateTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 (DEFAULT_TABLE table=test-table [id=b01801e0c99641f6be4714e9ebd8689f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.105729 23257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ac5f35cd8a464d8333c42b911eb216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.114737 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 (DEFAULT_TABLE table=test-table [id=b01801e0c99641f6be4714e9ebd8689f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.116175 23845 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Bootstrap starting.
I20260809 13:10:10.116250 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ac5f35cd8a464d8333c42b911eb216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.118156 23849 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Bootstrap starting.
I20260809 13:10:10.120096 23845 tablet_bootstrap.cc:654] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.121479 23845 log.cc:824] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:10.121625 23849 tablet_bootstrap.cc:654] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.122720 23849 log.cc:824] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:10.124490 23845 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: No bootstrap required, opened a new log
I20260809 13:10:10.124774 23845 ts_tablet_manager.cc:1397] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 13:10:10.125758 23849 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: No bootstrap required, opened a new log
I20260809 13:10:10.126034 23849 ts_tablet_manager.cc:1397] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260809 13:10:10.129091 23855 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Bootstrap starting.
I20260809 13:10:10.131664 23855 tablet_bootstrap.cc:654] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.132588 23855 log.cc:824] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:10.135320 23855 tablet_bootstrap.cc:492] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: No bootstrap required, opened a new log
I20260809 13:10:10.135188 23845 raft_consensus.cc:348] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.135572 23855 ts_tablet_manager.cc:1397] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 13:10:10.135706 23845 raft_consensus.cc:374] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.135828 23845 raft_consensus.cc:729] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b48667e68493090df6dad37603702, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.136207 23849 raft_consensus.cc:348] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.136626 23849 raft_consensus.cc:374] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.136477 23845 consensus_queue.cc:260] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.136725 23849 raft_consensus.cc:729] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.138288 23845 ts_tablet_manager.cc:1428] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260809 13:10:10.138795 23591 heartbeater.cc:503] Master 127.10.181.190:36861 was elected leader, sending a full tablet report...
I20260809 13:10:10.137409 23849 consensus_queue.cc:260] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.141847 23849 ts_tablet_manager.cc:1428] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260809 13:10:10.144316 23855 raft_consensus.cc:348] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.144646 23855 raft_consensus.cc:374] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.144742 23855 raft_consensus.cc:729] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313647ff35bb4b67b54734487abd8958, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.145252 23855 consensus_queue.cc:260] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.146234 23810 heartbeater.cc:503] Master 127.10.181.190:36861 was elected leader, sending a full tablet report...
I20260809 13:10:10.146626 23855 ts_tablet_manager.cc:1428] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260809 13:10:10.171967 23593 tablet.cc:2369] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:10:10.215718 23811 tablet.cc:2369] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:10:10.312542 23858 raft_consensus.cc:482] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:10.312881 23858 raft_consensus.cc:504] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.314989 23858 leader_election.cc:288] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd6b48667e68493090df6dad37603702 (127.10.181.130:34281), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:10.322752 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.315384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:48005 (local address 127.10.181.131:40159)
0809 13:10:10.315609 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.315616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.316287 (+ 671us) server_negotiation.cc:407] Connection header received
0809 13:10:10.316539 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.316546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.316629 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.316724 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.317423 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.318190 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.320883 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.321811 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.321973 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.322052 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.322356 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.322495 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 13:10:10.322836 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.315223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40159 (local address 127.10.181.129:48005)
0809 13:10:10.316184 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.316201 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.316386 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.316760 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.316765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.316792 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.317246 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.317261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.318326 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.318332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.320663 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.320679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.321996 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.322003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.322187 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.322222 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.322315 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.322441 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":57,"spinlock_wait_cycles":1792,"thread_start_us":734,"threads_started":1}
I20260809 13:10:10.323764 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.315764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:41665 (local address 127.10.181.130:34281)
0809 13:10:10.315970 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.315977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.316000 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:10:10.316303 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.316309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.316392 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.316475 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.317669 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.318782 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.321412 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.322651 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.322790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.322858 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.323393 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.323543 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:10:10.323841 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.315223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:34281 (local address 127.10.181.129:41665)
0809 13:10:10.315870 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.315890 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.316074 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.316918 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.316928 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.316996 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.317483 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.317498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.318997 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.319002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.321222 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.321238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.323069 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.323075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.323232 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.323263 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.323356 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.323475 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":26,"spinlock_wait_cycles":50432,"thread_start_us":308,"threads_started":1}
I20260809 13:10:10.324023 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ac5f35cd8a464d8333c42b911eb216" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958" is_pre_election: true
I20260809 13:10:10.324601 23729 raft_consensus.cc:2454] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 0.
I20260809 13:10:10.325307 23170 leader_election.cc:302] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, fbe09710ae154273a301d33cef7ca812; no voters:
I20260809 13:10:10.325325 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ac5f35cd8a464d8333c42b911eb216" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6b48667e68493090df6dad37603702" is_pre_election: true
I20260809 13:10:10.325774 23858 raft_consensus.cc:2781] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:10.325815 23520 raft_consensus.cc:2454] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 0.
I20260809 13:10:10.325940 23858 raft_consensus.cc:482] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:10.326052 23858 raft_consensus.cc:3037] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.329344 23858 raft_consensus.cc:504] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.330173 23858 leader_election.cc:288] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 election: Requested vote from peers bd6b48667e68493090df6dad37603702 (127.10.181.130:34281), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:10.330655 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ac5f35cd8a464d8333c42b911eb216" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958"
I20260809 13:10:10.330653 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ac5f35cd8a464d8333c42b911eb216" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6b48667e68493090df6dad37603702"
I20260809 13:10:10.330865 23729 raft_consensus.cc:3037] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.330865 23520 raft_consensus.cc:3037] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.334064 23520 raft_consensus.cc:2454] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 1.
I20260809 13:10:10.334086 23729 raft_consensus.cc:2454] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 1.
I20260809 13:10:10.334488 23171 leader_election.cc:302] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6b48667e68493090df6dad37603702, fbe09710ae154273a301d33cef7ca812; no voters:
I20260809 13:10:10.334844 23858 raft_consensus.cc:2781] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:10.335358 23858 raft_consensus.cc:686] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 1 LEADER]: Becoming Leader. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Running, Role: LEADER
I20260809 13:10:10.335983 23858 consensus_queue.cc:237] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.341071 23023 catalog_manager.cc:5495] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbe09710ae154273a301d33cef7ca812 (127.10.181.129). New cstate: current_term: 1 leader_uuid: "fbe09710ae154273a301d33cef7ca812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } health_report { overall_health: UNKNOWN } } }
W20260809 13:10:10.377873 23370 tablet.cc:2369] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:10:10.396242 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.391369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.10.181.129:44505)
0809 13:10:10.391563 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.391568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.391586 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:10.391714 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.391718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.391809 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.391927 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:10.392211 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.392923 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.394219 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.394607 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.394701 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.395037 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:10.395571 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:10.395723 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.395827 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.395918 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:10:10.404573 23520 raft_consensus.cc:1261] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Refusing update from remote peer fbe09710ae154273a301d33cef7ca812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:10.404588 23729 raft_consensus.cc:1261] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Refusing update from remote peer fbe09710ae154273a301d33cef7ca812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:10.405633 23858 consensus_queue.cc:1035] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:10.405977 23864 consensus_queue.cc:1035] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:10.418640 23878 mvcc.cc:205] Tried to move back new op lower bound from 7316607018607407104 to 7316607018340237312. Current Snapshot: MvccSnapshot[applied={T|T < 7316607018607407104}]
I20260809 13:10:10.435309 23023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:10.435703 23023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:10.440116 23023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Sending DeleteTablet for 3 replicas of tablet 89ac5f35cd8a464d8333c42b911eb216
I20260809 13:10:10.441362 23473 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:10 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34666
I20260809 13:10:10.441749 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:10 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57702
I20260809 13:10:10.442330 23257 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ac5f35cd8a464d8333c42b911eb216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:10 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49938
I20260809 13:10:10.442790 23023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:10:10.443550 23023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:10.448421 23889 tablet_replica.cc:330] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: stopping tablet replica
I20260809 13:10:10.448957 23889 raft_consensus.cc:2229] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:10.449555 23889 raft_consensus.cc:2258] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:10.452689 23472 tablet_service.cc:1467] Processing CreateTablet for tablet 77c00f81205241d1a423adab15f0864d (DEFAULT_TABLE table=test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.452824 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 77c00f81205241d1a423adab15f0864d (DEFAULT_TABLE table=test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.453299 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c00f81205241d1a423adab15f0864d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.453343 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c00f81205241d1a423adab15f0864d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.453882 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 77c00f81205241d1a423adab15f0864d (DEFAULT_TABLE table=test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:10.454344 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c00f81205241d1a423adab15f0864d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.460510 23855 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Bootstrap starting.
I20260809 13:10:10.460512 23849 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Bootstrap starting.
I20260809 13:10:10.462246 23855 tablet_bootstrap.cc:654] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.462291 23849 tablet_bootstrap.cc:654] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.463429 23845 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Bootstrap starting.
I20260809 13:10:10.464690 23889 ts_tablet_manager.cc:1905] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:10.464752 23890 tablet_replica.cc:330] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: stopping tablet replica
I20260809 13:10:10.465041 23845 tablet_bootstrap.cc:654] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.465399 23890 raft_consensus.cc:2229] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:10.466277 23890 raft_consensus.cc:2258] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:10.467119 23888 tablet_replica.cc:330] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: stopping tablet replica
I20260809 13:10:10.467577 23888 raft_consensus.cc:2229] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:10.468133 23888 raft_consensus.cc:2258] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:10.469456 23890 ts_tablet_manager.cc:1905] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:10.470093 23888 ts_tablet_manager.cc:1905] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:10.477140 23888 ts_tablet_manager.cc:1918] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:10.477372 23888 log.cc:1196] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/89ac5f35cd8a464d8333c42b911eb216
I20260809 13:10:10.478018 23888 ts_tablet_manager.cc:1939] T 89ac5f35cd8a464d8333c42b911eb216 P bd6b48667e68493090df6dad37603702: Deleting consensus metadata
I20260809 13:10:10.480149 23003 catalog_manager.cc:4841] TS bd6b48667e68493090df6dad37603702 (127.10.181.130:34281): tablet 89ac5f35cd8a464d8333c42b911eb216 (table test-table [id=b01801e0c99641f6be4714e9ebd8689f]) successfully deleted
I20260809 13:10:10.483821 23889 ts_tablet_manager.cc:1918] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:10.484009 23889 log.cc:1196] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/89ac5f35cd8a464d8333c42b911eb216
I20260809 13:10:10.484503 23889 ts_tablet_manager.cc:1939] T 89ac5f35cd8a464d8333c42b911eb216 P 313647ff35bb4b67b54734487abd8958: Deleting consensus metadata
I20260809 13:10:10.485663 23890 ts_tablet_manager.cc:1918] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:10.485895 23890 log.cc:1196] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/89ac5f35cd8a464d8333c42b911eb216
I20260809 13:10:10.485960 23845 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: No bootstrap required, opened a new log
I20260809 13:10:10.486111 23845 ts_tablet_manager.cc:1397] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.000s
I20260809 13:10:10.486557 23890 ts_tablet_manager.cc:1939] T 89ac5f35cd8a464d8333c42b911eb216 P fbe09710ae154273a301d33cef7ca812: Deleting consensus metadata
I20260809 13:10:10.486563 23002 catalog_manager.cc:4841] TS 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159): tablet 89ac5f35cd8a464d8333c42b911eb216 (table test-table [id=b01801e0c99641f6be4714e9ebd8689f]) successfully deleted
I20260809 13:10:10.487353 23845 raft_consensus.cc:348] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.488513 23003 catalog_manager.cc:4841] TS fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505): tablet 89ac5f35cd8a464d8333c42b911eb216 (table test-table [id=b01801e0c99641f6be4714e9ebd8689f]) successfully deleted
I20260809 13:10:10.488870 23855 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: No bootstrap required, opened a new log
I20260809 13:10:10.489027 23855 ts_tablet_manager.cc:1397] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.011s
I20260809 13:10:10.489739 23849 tablet_bootstrap.cc:492] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: No bootstrap required, opened a new log
I20260809 13:10:10.489902 23849 ts_tablet_manager.cc:1397] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.007s
I20260809 13:10:10.489913 23855 raft_consensus.cc:348] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.490142 23855 raft_consensus.cc:374] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.490239 23855 raft_consensus.cc:729] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313647ff35bb4b67b54734487abd8958, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.490499 23855 consensus_queue.cc:260] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.490664 23849 raft_consensus.cc:348] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.490887 23849 raft_consensus.cc:374] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.490984 23849 raft_consensus.cc:729] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.491217 23855 ts_tablet_manager.cc:1428] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 13:10:10.491268 23849 consensus_queue.cc:260] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.487597 23845 raft_consensus.cc:374] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.491588 23845 raft_consensus.cc:729] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b48667e68493090df6dad37603702, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.491858 23845 consensus_queue.cc:260] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.493001 23845 ts_tablet_manager.cc:1428] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 13:10:10.494537 23849 ts_tablet_manager.cc:1428] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 13:10:10.657663 23857 raft_consensus.cc:482] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:10.657902 23857 raft_consensus.cc:504] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.659487 23857 leader_election.cc:288] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:10.667914 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.660386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:35987 (local address 127.10.181.129:44505)
0809 13:10:10.660581 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.660588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.661589 (+ 1001us) server_negotiation.cc:407] Connection header received
0809 13:10:10.661822 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.661828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.661907 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.661964 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.662799 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.663679 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.665552 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.666812 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.666991 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.667071 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.667448 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.667579 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:10:10.667910 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.660183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:37989 (local address 127.10.181.131:40159)
0809 13:10:10.660388 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.660395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.661218 (+ 823us) server_negotiation.cc:407] Connection header received
0809 13:10:10.661517 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.661521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.661598 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.661681 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:10.662180 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.663036 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.665726 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.666870 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.666997 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.667071 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.667548 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.667672 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:10:10.667936 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.659839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:44505 (local address 127.10.181.130:35987)
0809 13:10:10.661433 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.661475 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.661646 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.662157 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.662161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.662192 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.662600 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.662611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.663828 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.663833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.665359 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.665372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.667052 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.667058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.667250 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.667292 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.667395 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.667540 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"mutex_wait_us":55,"thread_start_us":914,"threads_started":1}
I20260809 13:10:10.668644 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.659825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40159 (local address 127.10.181.130:37989)
0809 13:10:10.661097 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.661119 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.661339 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.661715 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.661719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.661751 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:10.662013 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.662022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.663177 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.663182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.665566 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.665581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.667228 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.667232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.667366 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.667398 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.668109 (+ 711us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.668249 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"mutex_wait_us":60,"thread_start_us":1278,"threads_started":1}
I20260809 13:10:10.669337 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c00f81205241d1a423adab15f0864d" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958" is_pre_election: true
I20260809 13:10:10.669306 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c00f81205241d1a423adab15f0864d" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812" is_pre_election: true
I20260809 13:10:10.669654 23729 raft_consensus.cc:2454] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:10.669978 23296 raft_consensus.cc:2454] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:10.670161 23404 leader_election.cc:302] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702; no voters:
I20260809 13:10:10.670593 23857 raft_consensus.cc:2781] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:10.670738 23857 raft_consensus.cc:482] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:10.670835 23857 raft_consensus.cc:3037] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.672952 23857 raft_consensus.cc:504] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.673600 23857 leader_election.cc:288] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Requested vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:10.674222 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c00f81205241d1a423adab15f0864d" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812"
I20260809 13:10:10.674290 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c00f81205241d1a423adab15f0864d" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958"
I20260809 13:10:10.674430 23296 raft_consensus.cc:3037] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.674481 23729 raft_consensus.cc:3037] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:10.676803 23729 raft_consensus.cc:2454] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:10.677201 23404 leader_election.cc:302] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702; no voters:
I20260809 13:10:10.677484 23857 raft_consensus.cc:2781] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:10.677731 23296 raft_consensus.cc:2454] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:10.677973 23857 raft_consensus.cc:686] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 1 LEADER]: Becoming Leader. State: Replica: bd6b48667e68493090df6dad37603702, State: Running, Role: LEADER
I20260809 13:10:10.678426 23857 consensus_queue.cc:237] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.682004 23024 catalog_manager.cc:5495] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6b48667e68493090df6dad37603702 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } health_report { overall_health: UNKNOWN } } }
Aug 09 13:10:10 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281008, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.190@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:10.731364 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.715768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.10.181.190:36861)
0809 13:10:10.716672 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:10.716678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:10.716698 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:10:10.716749 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:10.716753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:10.716843 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:10.716952 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:10.718492 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.719263 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.721050 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.721433 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.721564 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.725749 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:10.725755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:10.725758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:10.725789 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:10.728333 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:10.729243 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:10.729247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:10.729249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:10.729372 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:10.729814 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:10.729817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:10.729819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:10.730082 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:10.730196 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:10.730752 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 13:10:10.731076 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":775,"thread_start_us":578,"threads_started":1}
I20260809 13:10:10.731401 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:10.715706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:36861 (local address 127.0.0.1:41486)
0809 13:10:10.716359 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:10.716385 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:10.716571 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:10.717055 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:10.717061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:10.718074 (+ 1013us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:10.718327 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.718341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.719430 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.719436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.720897 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:10.720915 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:10.721655 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:10.721663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:10.721858 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:10.722584 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:10.722612 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:10.725558 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:10.728491 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:10.728505 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:10.728513 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:10.729116 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:10.729512 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:10.729517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:10.729521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:10.729700 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:10.730238 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:10.730247 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:10.730452 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 13:10:10.730658 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 13:10:10.730971 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":337,"threads_started":1}
I20260809 13:10:10.737135 23024 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:41486:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 13:10:10.738233 23024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:10.745301 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 9cffeb2068ad4691b3d34e2749c3e181 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cda0cbe2b804778a857dba4ce20b47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:10:10.745379 23473 tablet_service.cc:1467] Processing CreateTablet for tablet 9cffeb2068ad4691b3d34e2749c3e181 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cda0cbe2b804778a857dba4ce20b47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:10:10.745934 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cffeb2068ad4691b3d34e2749c3e181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.746016 23473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cffeb2068ad4691b3d34e2749c3e181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.750275 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 9cffeb2068ad4691b3d34e2749c3e181 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cda0cbe2b804778a857dba4ce20b47f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 13:10:10.750845 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cffeb2068ad4691b3d34e2749c3e181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:10.752687 23849 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: Bootstrap starting.
I20260809 13:10:10.753301 23845 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: Bootstrap starting.
I20260809 13:10:10.754534 23849 tablet_bootstrap.cc:654] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.754971 23845 tablet_bootstrap.cc:654] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.758116 23855 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: Bootstrap starting.
I20260809 13:10:10.759816 23855 tablet_bootstrap.cc:654] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:10.760435 23845 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: No bootstrap required, opened a new log
I20260809 13:10:10.760689 23845 ts_tablet_manager.cc:1397] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260809 13:10:10.761547 23845 raft_consensus.cc:348] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.761951 23845 raft_consensus.cc:374] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.762072 23845 raft_consensus.cc:729] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b48667e68493090df6dad37603702, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.762377 23845 consensus_queue.cc:260] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.762887 23857 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:10.763082 23845 ts_tablet_manager.cc:1428] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 13:10:10.763206 23857 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:10.765213 23849 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: No bootstrap required, opened a new log
I20260809 13:10:10.765493 23849 ts_tablet_manager.cc:1397] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.008s
I20260809 13:10:10.766285 23849 raft_consensus.cc:348] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.766497 23849 raft_consensus.cc:374] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.766592 23849 raft_consensus.cc:729] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.766853 23849 consensus_queue.cc:260] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.767469 23849 ts_tablet_manager.cc:1428] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 13:10:10.767307 23858 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:10.768316 23858 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:10.770830 23855 tablet_bootstrap.cc:492] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: No bootstrap required, opened a new log
I20260809 13:10:10.771021 23855 ts_tablet_manager.cc:1397] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20260809 13:10:10.771785 23855 raft_consensus.cc:348] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.772039 23855 raft_consensus.cc:374] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:10.772132 23855 raft_consensus.cc:729] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313647ff35bb4b67b54734487abd8958, State: Initialized, Role: FOLLOWER
I20260809 13:10:10.772389 23855 consensus_queue.cc:260] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:10.772877 23859 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:10.773240 23859 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:10.773741 23855 ts_tablet_manager.cc:1428] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:11.074497 23857 consensus_queue.cc:1035] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.079752 23857 consensus_queue.cc:1035] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.094559 23857 raft_consensus.cc:482] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:11.094745 23857 raft_consensus.cc:504] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.095240 23857 leader_election.cc:288] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.095645 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cffeb2068ad4691b3d34e2749c3e181" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812" is_pre_election: true
I20260809 13:10:11.095687 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cffeb2068ad4691b3d34e2749c3e181" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958" is_pre_election: true
I20260809 13:10:11.095860 23296 raft_consensus.cc:2454] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:11.095888 23729 raft_consensus.cc:2454] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:11.096269 23405 leader_election.cc:302] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702, fbe09710ae154273a301d33cef7ca812; no voters:
I20260809 13:10:11.096551 23857 raft_consensus.cc:2781] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:11.096653 23857 raft_consensus.cc:482] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:11.096760 23857 raft_consensus.cc:3037] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.098807 23857 raft_consensus.cc:504] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.099418 23857 leader_election.cc:288] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Requested vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.099779 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cffeb2068ad4691b3d34e2749c3e181" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812"
I20260809 13:10:11.099810 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cffeb2068ad4691b3d34e2749c3e181" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958"
I20260809 13:10:11.099997 23296 raft_consensus.cc:3037] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.099999 23729 raft_consensus.cc:3037] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.102147 23729 raft_consensus.cc:2454] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:11.102180 23296 raft_consensus.cc:2454] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:11.102517 23404 leader_election.cc:302] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702; no voters:
I20260809 13:10:11.102772 23857 raft_consensus.cc:2781] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:11.102950 23857 raft_consensus.cc:686] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [term 1 LEADER]: Becoming Leader. State: Replica: bd6b48667e68493090df6dad37603702, State: Running, Role: LEADER
I20260809 13:10:11.103217 23857 consensus_queue.cc:237] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.104688 23896 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: TxnStatusTablet state changed. Reason: New leader bd6b48667e68493090df6dad37603702. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.104998 23896 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:10:11.105626 23922 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 13:10:11.105858 23922 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 13:10:11.106107 23024 catalog_manager.cc:5495] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6b48667e68493090df6dad37603702 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260809 13:10:11.132066 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.125721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.10.181.130:34281)
0809 13:10:11.126294 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.126298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.126318 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:10:11.126392 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.126395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.126474 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.126600 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:11.127051 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.127902 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.129813 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.130249 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.130343 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.130896 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.131456 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.131564 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.131727 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.131792 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":316,"threads_started":1}
I20260809 13:10:11.132189 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.125568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:34281 (local address 127.0.0.1:34682)
0809 13:10:11.125845 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:11.125867 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:11.126009 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:11.126660 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:11.126666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:11.126679 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:10:11.126883 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:11.126894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.128041 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.128047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:11.129660 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:11.129676 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.130414 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.130421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:11.130629 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.130702 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.131575 (+ 873us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.131600 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:10:11.131692 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:10:11.131854 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 13:10:11.136109 23296 raft_consensus.cc:1261] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Refusing update from remote peer bd6b48667e68493090df6dad37603702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.136109 23729 raft_consensus.cc:1261] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Refusing update from remote peer bd6b48667e68493090df6dad37603702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.136727 23857 consensus_queue.cc:1035] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.137012 23896 consensus_queue.cc:1035] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.140707 23858 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: TxnStatusTablet state changed. Reason: New leader bd6b48667e68493090df6dad37603702. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.140995 23858 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:11.141937 23859 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: TxnStatusTablet state changed. Reason: New leader bd6b48667e68493090df6dad37603702. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.142217 23859 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:11.144411 23917 mvcc.cc:205] Tried to move back new op lower bound from 7316607021606793216 to 7316607021480812544. Current Snapshot: MvccSnapshot[applied={T|T < 7316607021606793216}]
I20260809 13:10:11.145681 23857 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.145992 23857 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:10:11.147136 23896 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.147405 23896 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P bd6b48667e68493090df6dad37603702: This TxnStatusTablet replica's current role is: LEADER
I20260809 13:10:11.154433 23859 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.154680 23859 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P 313647ff35bb4b67b54734487abd8958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:11.162485 23858 tablet_replica.cc:439] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } } }
I20260809 13:10:11.162729 23858 tablet_replica.cc:442] T 9cffeb2068ad4691b3d34e2749c3e181 P fbe09710ae154273a301d33cef7ca812: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 13:10:11.169799 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.163572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.10.181.130:34281)
0809 13:10:11.164057 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.164062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.164269 (+ 207us) server_negotiation.cc:407] Connection header received
0809 13:10:11.164557 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.164562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.164656 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.164760 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:11.165144 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.166320 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.167636 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.168359 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.168504 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.168978 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.169103 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.169198 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.169406 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.169594 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368}
I20260809 13:10:11.184763 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.179564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:52471 (local address 127.10.181.130:34281)
0809 13:10:11.179709 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.179713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.180142 (+ 429us) server_negotiation.cc:407] Connection header received
0809 13:10:11.180324 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.180327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.180390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.180464 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:11.180948 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.181744 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.183173 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.183582 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.183685 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.184077 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.184164 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.184239 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.184390 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.184454 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 13:10:11.184839 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.179481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:34281 (local address 127.10.181.130:52471)
0809 13:10:11.180043 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:11.180057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:11.180185 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:11.180498 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:11.180502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:11.180513 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 13:10:11.180789 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:11.180798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.181880 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.181885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:11.183046 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:11.183060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.183725 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.183731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:11.183856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.183907 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.184263 (+ 356us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.184282 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 13:10:11.184353 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 13:10:11.184485 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":339,"threads_started":1}
I20260809 13:10:11.243669 23024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.243920 23024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.247136 23024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Sending DeleteTablet for 3 replicas of tablet 77c00f81205241d1a423adab15f0864d
I20260809 13:10:11.248031 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c00f81205241d1a423adab15f0864d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57702
I20260809 13:10:11.248420 23256 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c00f81205241d1a423adab15f0864d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49938
I20260809 13:10:11.248664 23958 tablet_replica.cc:330] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: stopping tablet replica
I20260809 13:10:11.248890 23958 raft_consensus.cc:2229] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.249145 23958 raft_consensus.cc:2258] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.250176 23958 ts_tablet_manager.cc:1905] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.253831 23472 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c00f81205241d1a423adab15f0864d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34666
I20260809 13:10:11.257171 23958 ts_tablet_manager.cc:1918] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.257335 23958 log.cc:1196] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/77c00f81205241d1a423adab15f0864d
I20260809 13:10:11.257848 23958 ts_tablet_manager.cc:1939] T 77c00f81205241d1a423adab15f0864d P 313647ff35bb4b67b54734487abd8958: Deleting consensus metadata
I20260809 13:10:11.258837 23002 catalog_manager.cc:4841] TS 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159): tablet 77c00f81205241d1a423adab15f0864d (table test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]) successfully deleted
I20260809 13:10:11.261808 23959 tablet_replica.cc:330] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: stopping tablet replica
I20260809 13:10:11.261992 23959 raft_consensus.cc:2229] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.262207 23959 raft_consensus.cc:2258] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.263064 23959 ts_tablet_manager.cc:1905] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.268970 23959 ts_tablet_manager.cc:1918] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.269101 23959 log.cc:1196] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/77c00f81205241d1a423adab15f0864d
I20260809 13:10:11.269524 23959 ts_tablet_manager.cc:1939] T 77c00f81205241d1a423adab15f0864d P fbe09710ae154273a301d33cef7ca812: Deleting consensus metadata
I20260809 13:10:11.270507 23003 catalog_manager.cc:4841] TS fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505): tablet 77c00f81205241d1a423adab15f0864d (table test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]) successfully deleted
I20260809 13:10:11.272890 23960 tablet_replica.cc:330] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: stopping tablet replica
I20260809 13:10:11.273147 23960 raft_consensus.cc:2229] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:11.273809 23960 raft_consensus.cc:2258] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.274887 23960 ts_tablet_manager.cc:1905] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.281780 23960 ts_tablet_manager.cc:1918] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.281924 23960 log.cc:1196] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/77c00f81205241d1a423adab15f0864d
I20260809 13:10:11.282330 23960 ts_tablet_manager.cc:1939] T 77c00f81205241d1a423adab15f0864d P bd6b48667e68493090df6dad37603702: Deleting consensus metadata
I20260809 13:10:11.283495 23003 catalog_manager.cc:4841] TS bd6b48667e68493090df6dad37603702 (127.10.181.130:34281): tablet 77c00f81205241d1a423adab15f0864d (table test-table [id=89ed8e355e15491d8f4fa8adf3f549b1]) successfully deleted
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 13:10:11.326489 23968 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:11.315060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.10.181.190:36861)
0809 13:10:11.315842 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.315847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.315866 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:10:11.315923 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.315927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.315989 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.316092 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:11.317786 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.318572 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.320372 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.320762 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.320853 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.326193 (+ 5340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41500: BlockingRecv error: recv got EOF from 127.0.0.1:41500 (error 108)
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":552,"threads_started":1}
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM
I20260809 13:10:11.343488 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.329306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.10.181.190:36861)
0809 13:10:11.329521 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.329526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.329809 (+ 283us) server_negotiation.cc:407] Connection header received
0809 13:10:11.329935 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.329940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.330015 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.330111 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:11.331133 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.332224 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.333384 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.333973 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.334089 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.337717 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:11.337725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:11.337729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:11.337770 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:11.340984 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.341584 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.341590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.341594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.341755 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.342100 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.342105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.342109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.342460 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:11.342712 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.342915 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.343134 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:10:11.347041 23024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:10:11.347666 23024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:11.354583 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (DEFAULT_TABLE table=test-table [id=c28663c6d97f4523b7890946932f28a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.354707 23472 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (DEFAULT_TABLE table=test-table [id=c28663c6d97f4523b7890946932f28a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.355091 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd75aeff8841e0af58c78bbc9c2f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.355149 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd75aeff8841e0af58c78bbc9c2f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.360642 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (DEFAULT_TABLE table=test-table [id=c28663c6d97f4523b7890946932f28a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.360916 23979 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Bootstrap starting.
I20260809 13:10:11.361066 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd75aeff8841e0af58c78bbc9c2f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.362150 23979 tablet_bootstrap.cc:654] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.365566 23979 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: No bootstrap required, opened a new log
I20260809 13:10:11.365780 23979 ts_tablet_manager.cc:1397] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 13:10:11.366036 23983 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Bootstrap starting.
I20260809 13:10:11.367987 23983 tablet_bootstrap.cc:654] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.368309 23979 raft_consensus.cc:348] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.368641 23979 raft_consensus.cc:374] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.368785 23979 raft_consensus.cc:729] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.369127 23979 consensus_queue.cc:260] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.369874 23979 ts_tablet_manager.cc:1428] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:11.371642 23983 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: No bootstrap required, opened a new log
I20260809 13:10:11.371806 23983 ts_tablet_manager.cc:1397] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260809 13:10:11.372634 23983 raft_consensus.cc:348] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.372906 23983 raft_consensus.cc:374] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.373046 23983 raft_consensus.cc:729] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b48667e68493090df6dad37603702, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.373312 23983 consensus_queue.cc:260] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.374022 23983 ts_tablet_manager.cc:1428] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 13:10:11.376688 23987 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Bootstrap starting.
I20260809 13:10:11.378417 23987 tablet_bootstrap.cc:654] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.380615 23987 tablet_bootstrap.cc:492] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: No bootstrap required, opened a new log
I20260809 13:10:11.380770 23987 ts_tablet_manager.cc:1397] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 13:10:11.381608 23987 raft_consensus.cc:348] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.381853 23987 raft_consensus.cc:374] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.381953 23987 raft_consensus.cc:729] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313647ff35bb4b67b54734487abd8958, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.382189 23987 consensus_queue.cc:260] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.382778 23987 ts_tablet_manager.cc:1428] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:10:11.450241 23858 raft_consensus.cc:482] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:11.450420 23858 raft_consensus.cc:504] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.451181 23858 leader_election.cc:288] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd6b48667e68493090df6dad37603702 (127.10.181.130:34281), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.451740 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd75aeff8841e0af58c78bbc9c2f47" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6b48667e68493090df6dad37603702" is_pre_election: true
I20260809 13:10:11.451771 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd75aeff8841e0af58c78bbc9c2f47" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958" is_pre_election: true
I20260809 13:10:11.452037 23729 raft_consensus.cc:2454] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 0.
I20260809 13:10:11.452050 23520 raft_consensus.cc:2454] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 0.
I20260809 13:10:11.452633 23171 leader_election.cc:302] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6b48667e68493090df6dad37603702, fbe09710ae154273a301d33cef7ca812; no voters:
I20260809 13:10:11.452988 23858 raft_consensus.cc:2781] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:11.453088 23858 raft_consensus.cc:482] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:11.453183 23858 raft_consensus.cc:3037] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.455449 23858 raft_consensus.cc:504] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.456101 23858 leader_election.cc:288] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 election: Requested vote from peers bd6b48667e68493090df6dad37603702 (127.10.181.130:34281), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.456602 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd75aeff8841e0af58c78bbc9c2f47" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958"
I20260809 13:10:11.456681 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd75aeff8841e0af58c78bbc9c2f47" candidate_uuid: "fbe09710ae154273a301d33cef7ca812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6b48667e68493090df6dad37603702"
I20260809 13:10:11.456837 23729 raft_consensus.cc:3037] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.456888 23520 raft_consensus.cc:3037] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.459712 23729 raft_consensus.cc:2454] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 1.
I20260809 13:10:11.459911 23520 raft_consensus.cc:2454] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbe09710ae154273a301d33cef7ca812 in term 1.
I20260809 13:10:11.460307 23170 leader_election.cc:302] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, fbe09710ae154273a301d33cef7ca812; no voters:
I20260809 13:10:11.460664 23858 raft_consensus.cc:2781] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:11.461050 23858 raft_consensus.cc:686] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 1 LEADER]: Becoming Leader. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Running, Role: LEADER
I20260809 13:10:11.461395 23858 consensus_queue.cc:237] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.464149 23023 catalog_manager.cc:5495] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbe09710ae154273a301d33cef7ca812 (127.10.181.129). New cstate: current_term: 1 leader_uuid: "fbe09710ae154273a301d33cef7ca812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260809 13:10:11.490593 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.485137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.10.181.129:44505)
0809 13:10:11.485750 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.485754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.485772 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:11.485824 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.485827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.485886 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.485973 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:11.486401 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.487265 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.488740 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.489181 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.489310 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.489866 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.489974 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.490052 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.490233 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.490310 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":375,"threads_started":1}
I20260809 13:10:11.493252 23729 raft_consensus.cc:1261] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Refusing update from remote peer fbe09710ae154273a301d33cef7ca812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.493333 23520 raft_consensus.cc:1261] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Refusing update from remote peer fbe09710ae154273a301d33cef7ca812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.493925 23993 consensus_queue.cc:1035] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.494307 23858 consensus_queue.cc:1035] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.509378 23024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.509642 23024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.512483 23024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Sending DeleteTablet for 3 replicas of tablet 0ccd75aeff8841e0af58c78bbc9c2f47
I20260809 13:10:11.513232 23472 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34666
I20260809 13:10:11.513309 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57702
I20260809 13:10:11.513504 23960 tablet_replica.cc:330] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: stopping tablet replica
I20260809 13:10:11.513587 23958 tablet_replica.cc:330] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: stopping tablet replica
I20260809 13:10:11.513722 23960 raft_consensus.cc:2229] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.513697 23256 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd75aeff8841e0af58c78bbc9c2f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49938
I20260809 13:10:11.513779 23958 raft_consensus.cc:2229] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.513947 23959 tablet_replica.cc:330] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: stopping tablet replica
I20260809 13:10:11.513938 23960 raft_consensus.cc:2258] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.513986 23958 raft_consensus.cc:2258] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.514137 23959 raft_consensus.cc:2229] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:11.514403 23959 raft_consensus.cc:2258] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.514986 23958 ts_tablet_manager.cc:1905] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.514976 23960 ts_tablet_manager.cc:1905] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.515388 23959 ts_tablet_manager.cc:1905] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.521405 23958 ts_tablet_manager.cc:1918] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:11.521490 23960 ts_tablet_manager.cc:1918] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:11.521656 23960 log.cc:1196] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/0ccd75aeff8841e0af58c78bbc9c2f47
I20260809 13:10:11.521667 23958 log.cc:1196] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/0ccd75aeff8841e0af58c78bbc9c2f47
I20260809 13:10:11.522158 23958 ts_tablet_manager.cc:1939] T 0ccd75aeff8841e0af58c78bbc9c2f47 P 313647ff35bb4b67b54734487abd8958: Deleting consensus metadata
I20260809 13:10:11.522158 23960 ts_tablet_manager.cc:1939] T 0ccd75aeff8841e0af58c78bbc9c2f47 P bd6b48667e68493090df6dad37603702: Deleting consensus metadata
I20260809 13:10:11.523205 23002 catalog_manager.cc:4841] TS 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159): tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (table test-table [id=c28663c6d97f4523b7890946932f28a9]) successfully deleted
I20260809 13:10:11.522851 23024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 13:10:11.523442 23003 catalog_manager.cc:4841] TS bd6b48667e68493090df6dad37603702 (127.10.181.130:34281): tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (table test-table [id=c28663c6d97f4523b7890946932f28a9]) successfully deleted
W20260809 13:10:11.523733 23024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:11.524878 23959 ts_tablet_manager.cc:1918] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:11.525034 23959 log.cc:1196] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/0ccd75aeff8841e0af58c78bbc9c2f47
I20260809 13:10:11.525465 23959 ts_tablet_manager.cc:1939] T 0ccd75aeff8841e0af58c78bbc9c2f47 P fbe09710ae154273a301d33cef7ca812: Deleting consensus metadata
I20260809 13:10:11.526587 23003 catalog_manager.cc:4841] TS fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505): tablet 0ccd75aeff8841e0af58c78bbc9c2f47 (table test-table [id=c28663c6d97f4523b7890946932f28a9]) successfully deleted
I20260809 13:10:11.530442 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7de9b6646643fe81b3a5031da57047 (DEFAULT_TABLE table=test-table [id=4da5c299994a4284851e91685892e0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.530859 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7de9b6646643fe81b3a5031da57047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.530874 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7de9b6646643fe81b3a5031da57047 (DEFAULT_TABLE table=test-table [id=4da5c299994a4284851e91685892e0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.531263 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7de9b6646643fe81b3a5031da57047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.531296 23472 tablet_service.cc:1467] Processing CreateTablet for tablet 9c7de9b6646643fe81b3a5031da57047 (DEFAULT_TABLE table=test-table [id=4da5c299994a4284851e91685892e0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:11.531699 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c7de9b6646643fe81b3a5031da57047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:11.537358 23979 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Bootstrap starting.
I20260809 13:10:11.538651 23983 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Bootstrap starting.
I20260809 13:10:11.538972 23979 tablet_bootstrap.cc:654] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.540228 23983 tablet_bootstrap.cc:654] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.540748 23979 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: No bootstrap required, opened a new log
I20260809 13:10:11.540915 23979 ts_tablet_manager.cc:1397] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 13:10:11.541754 23979 raft_consensus.cc:348] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.541980 23979 raft_consensus.cc:374] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.542037 23979 raft_consensus.cc:729] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe09710ae154273a301d33cef7ca812, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.542250 23979 consensus_queue.cc:260] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.542743 23979 ts_tablet_manager.cc:1428] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:10:11.545112 23987 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Bootstrap starting.
I20260809 13:10:11.546658 23987 tablet_bootstrap.cc:654] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:11.548638 23987 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: No bootstrap required, opened a new log
I20260809 13:10:11.548843 23987 ts_tablet_manager.cc:1397] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 13:10:11.549597 23987 raft_consensus.cc:348] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.549845 23987 raft_consensus.cc:374] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.549922 23987 raft_consensus.cc:729] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313647ff35bb4b67b54734487abd8958, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.550146 23987 consensus_queue.cc:260] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.550720 23987 ts_tablet_manager.cc:1428] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 13:10:11.551271 23983 tablet_bootstrap.cc:492] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: No bootstrap required, opened a new log
I20260809 13:10:11.551451 23983 ts_tablet_manager.cc:1397] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260809 13:10:11.552265 23983 raft_consensus.cc:348] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.552526 23983 raft_consensus.cc:374] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:11.552621 23983 raft_consensus.cc:729] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b48667e68493090df6dad37603702, State: Initialized, Role: FOLLOWER
I20260809 13:10:11.552858 23983 consensus_queue.cc:260] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.553367 23983 ts_tablet_manager.cc:1428] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 13:10:11.557116 23950 raft_consensus.cc:482] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:11.557288 23950 raft_consensus.cc:504] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.558022 23950 leader_election.cc:288] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.558399 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7de9b6646643fe81b3a5031da57047" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958" is_pre_election: true
I20260809 13:10:11.558454 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7de9b6646643fe81b3a5031da57047" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812" is_pre_election: true
I20260809 13:10:11.558573 23729 raft_consensus.cc:2454] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:11.558668 23296 raft_consensus.cc:2454] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 0.
I20260809 13:10:11.559000 23404 leader_election.cc:302] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702; no voters:
I20260809 13:10:11.559276 23950 raft_consensus.cc:2781] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:11.559414 23950 raft_consensus.cc:482] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:11.559505 23950 raft_consensus.cc:3037] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.562340 23950 raft_consensus.cc:504] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.562908 23950 leader_election.cc:288] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Requested vote from peers fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505), 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159)
I20260809 13:10:11.563305 23729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7de9b6646643fe81b3a5031da57047" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313647ff35bb4b67b54734487abd8958"
I20260809 13:10:11.563337 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c7de9b6646643fe81b3a5031da57047" candidate_uuid: "bd6b48667e68493090df6dad37603702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbe09710ae154273a301d33cef7ca812"
I20260809 13:10:11.563485 23729 raft_consensus.cc:3037] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.563546 23296 raft_consensus.cc:3037] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:11.565392 23729 raft_consensus.cc:2454] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:11.565824 23404 leader_election.cc:302] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313647ff35bb4b67b54734487abd8958, bd6b48667e68493090df6dad37603702; no voters:
I20260809 13:10:11.566254 23296 raft_consensus.cc:2454] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b48667e68493090df6dad37603702 in term 1.
I20260809 13:10:11.566561 23950 raft_consensus.cc:2781] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:11.566788 23950 raft_consensus.cc:686] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 1 LEADER]: Becoming Leader. State: Replica: bd6b48667e68493090df6dad37603702, State: Running, Role: LEADER
I20260809 13:10:11.567046 23950 consensus_queue.cc:237] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } }
I20260809 13:10:11.569904 23023 catalog_manager.cc:5495] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6b48667e68493090df6dad37603702 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "bd6b48667e68493090df6dad37603702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd6b48667e68493090df6dad37603702" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260809 13:10:11.594815 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.589340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.10.181.130:34281)
0809 13:10:11.589546 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.589552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.589569 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:10:11.589736 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.589740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.589802 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.589872 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:11.590313 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.591041 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.592993 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.593478 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.593588 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.594095 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:11.594208 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:11.594275 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.594447 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.594511 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 13:10:11.603029 23729 raft_consensus.cc:1261] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Refusing update from remote peer bd6b48667e68493090df6dad37603702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.603228 23296 raft_consensus.cc:1261] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Refusing update from remote peer bd6b48667e68493090df6dad37603702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:11.603714 23950 consensus_queue.cc:1035] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313647ff35bb4b67b54734487abd8958" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.603979 23857 consensus_queue.cc:1035] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbe09710ae154273a301d33cef7ca812" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:11.619324 23919 mvcc.cc:205] Tried to move back new op lower bound from 7316607023521296384 to 7316607023380803584. Current Snapshot: MvccSnapshot[applied={T|T < 7316607023521296384}]
I20260809 13:10:11.665079 23023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.665314 23023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:11.668020 23023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b223145cfec4244bac9de369b4b381a: Sending DeleteTablet for 3 replicas of tablet 9c7de9b6646643fe81b3a5031da57047
I20260809 13:10:11.668882 23472 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7de9b6646643fe81b3a5031da57047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34666
I20260809 13:10:11.669029 23256 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7de9b6646643fe81b3a5031da57047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49938
I20260809 13:10:11.669183 23960 tablet_replica.cc:330] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: stopping tablet replica
I20260809 13:10:11.669356 23959 tablet_replica.cc:330] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: stopping tablet replica
I20260809 13:10:11.669328 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c7de9b6646643fe81b3a5031da57047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:11 UTC) from {username='oryx', principal='oryx/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57702
I20260809 13:10:11.669373 23960 raft_consensus.cc:2229] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:11.669541 23959 raft_consensus.cc:2229] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.669737 23960 raft_consensus.cc:2258] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.669752 23959 raft_consensus.cc:2258] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.669788 23958 tablet_replica.cc:330] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: stopping tablet replica
I20260809 13:10:11.669972 23958 raft_consensus.cc:2229] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:11.670149 23958 raft_consensus.cc:2258] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:11.671262 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 23141
I20260809 13:10:11.671420 23959 ts_tablet_manager.cc:1905] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.671595 23958 ts_tablet_manager.cc:1905] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.674459 23960 ts_tablet_manager.cc:1905] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:11.678637 23958 ts_tablet_manager.cc:1918] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.678756 23958 log.cc:1196] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/9c7de9b6646643fe81b3a5031da57047
I20260809 13:10:11.679152 23958 ts_tablet_manager.cc:1939] T 9c7de9b6646643fe81b3a5031da57047 P 313647ff35bb4b67b54734487abd8958: Deleting consensus metadata
I20260809 13:10:11.680302 23002 catalog_manager.cc:4841] TS 313647ff35bb4b67b54734487abd8958 (127.10.181.131:40159): tablet 9c7de9b6646643fe81b3a5031da57047 (table test-table [id=4da5c299994a4284851e91685892e0bb]) successfully deleted
I20260809 13:10:11.683570 23959 ts_tablet_manager.cc:1918] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.683725 23959 log.cc:1196] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/9c7de9b6646643fe81b3a5031da57047
I20260809 13:10:11.684110 23959 ts_tablet_manager.cc:1939] T 9c7de9b6646643fe81b3a5031da57047 P fbe09710ae154273a301d33cef7ca812: Deleting consensus metadata
I20260809 13:10:11.684983 23003 catalog_manager.cc:4841] TS fbe09710ae154273a301d33cef7ca812 (127.10.181.129:44505): tablet 9c7de9b6646643fe81b3a5031da57047 (table test-table [id=4da5c299994a4284851e91685892e0bb]) successfully deleted
I20260809 13:10:11.687577 23960 ts_tablet_manager.cc:1918] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 13:10:11.687788 23960 log.cc:1196] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/9c7de9b6646643fe81b3a5031da57047
I20260809 13:10:11.688271 23960 ts_tablet_manager.cc:1939] T 9c7de9b6646643fe81b3a5031da57047 P bd6b48667e68493090df6dad37603702: Deleting consensus metadata
I20260809 13:10:11.689504 23003 catalog_manager.cc:4841] TS bd6b48667e68493090df6dad37603702 (127.10.181.130:34281): tablet 9c7de9b6646643fe81b3a5031da57047 (table test-table [id=4da5c299994a4284851e91685892e0bb]) successfully deleted
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.129@KRBTEST.COM
I20260809 13:10:11.698199 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.673631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.10.181.129:44505)
0809 13:10:11.673797 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.673803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.681989 (+ 8186us) server_negotiation.cc:407] Connection header received
0809 13:10:11.682202 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.682207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.682281 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.682379 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:11.683693 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.684508 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.686156 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.686800 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.686926 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.691731 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:11.691751 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:11.691774 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:11.691813 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:11.695450 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.696121 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.696131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.696135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.696290 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.696625 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.696630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.696634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.697084 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:11.697405 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.697626 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.697847 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:10:11.698982 23361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:11.823071 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 23141
I20260809 13:10:11.857606 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 23373
Aug 09 13:10:11 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.130@KRBTEST.COM
I20260809 13:10:11.875439 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:11.858439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.10.181.130:34281)
0809 13:10:11.858619 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:11.858624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:11.858697 (+ 73us) server_negotiation.cc:407] Connection header received
0809 13:10:11.858883 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:11.858887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:11.858950 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:11.859039 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:11.860677 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.861552 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.863391 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:11.863833 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:11.863932 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:11.869139 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:11.869156 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:11.869174 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:11.869208 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:11.872362 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.873188 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.873195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.873198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.873342 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:11.873764 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:11.873767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:11.873769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:11.874301 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:11.874543 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:11.874894 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 13:10:11.875057 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:10:11.876168 23580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:12.029692 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 23373
I20260809 13:10:12.061712 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 23599
Aug 09 13:10:12 dist-test-slave-42z9 krb5kdc[22950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.181.131@KRBTEST.COM
I20260809 13:10:12.075970 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:12.062544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.10.181.131:40159)
0809 13:10:12.062975 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:12.062980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:12.062997 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:10:12.063054 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:12.063059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:12.063125 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:12.063219 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:12.064432 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:12.065157 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:12.066471 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:12.066895 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:12.066987 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:12.071046 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:12.071061 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:12.071079 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:12.071120 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:12.073794 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:12.074355 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:12.074362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:12.074364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:12.074482 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:12.074769 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:12.074772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:12.074773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:12.075107 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:12.075310 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:12.075535 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 13:10:12.075716 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":221,"threads_started":1}
I20260809 13:10:12.076598 23801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:12.172472 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 23599
I20260809 13:10:12.200306 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 22968
I20260809 13:10:12.201474 23087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:12.316197 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 22968
2026-08-09T13:10:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5996 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:10:12 dist-test-slave-42z9 krb5kdc[24023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:10:12 dist-test-slave-42z9 krb5kdc[24023](info): set up 2 sockets
Aug 09 13:10:12 dist-test-slave-42z9 krb5kdc[24023](info): commencing operation
krb5kdc: starting...
W20260809 13:10:13.522578 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:10:13 dist-test-slave-42z9 krb5kdc[24023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:10:13.760435 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37351
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:37351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:14.087945 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:14.088213 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:14.088308 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:14.095667 24045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:14.095775 24045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:14.095820 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:14.095876 24045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:14.095922 24045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:14.105834 24045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:37351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:14.107084 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:14.108359 24045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:14.119570 24055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:14.119544 24053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:14.119625 24052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:14.120215 24045 server_base.cc:1034] running on GCE node
I20260809 13:10:14.120975 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:14.122346 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:14.123600 24045 hybrid_clock.cc:648] HybridClock initialized: now 1786281014123537 us; error 44 us; skew 500 ppm
I20260809 13:10:14.123899 24045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T13:10:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1874 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:10:14 dist-test-slave-42z9 krb5kdc[24066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:10:14 dist-test-slave-42z9 krb5kdc[24066](info): set up 2 sockets
Aug 09 13:10:14 dist-test-slave-42z9 krb5kdc[24066](info): commencing operation
krb5kdc: starting...
W20260809 13:10:15.414474 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:10:15 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:10:15.658136 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34049
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:15.979919 24083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:15.980211 24083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:15.980355 24083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:15.987257 24083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:15.987365 24083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:15.987443 24083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:15.987519 24083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:15.987581 24083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:15.997372 24083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:15.998651 24083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:15.999909 24083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:16.011003 24089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:16.010957 24093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:16.011037 24090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:16.011878 24083 server_base.cc:1034] running on GCE node
I20260809 13:10:16.012588 24083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:16.013942 24083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:16.015147 24083 hybrid_clock.cc:648] HybridClock initialized: now 1786281016015107 us; error 29 us; skew 500 ppm
I20260809 13:10:16.015456 24083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:16 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281016, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:16.022059 24083 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:10:16.023434 24083 webserver.cc:460] Webserver started at http://127.10.181.190:38951/ using document root <none> and password file <none>
I20260809 13:10:16.024067 24083 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:16.024166 24083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:16.024420 24083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:16.027309 24083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "b88f20e3dd7a470c9660eff0d0e49c52"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "2d256cbbcd34650cc55941862b0ca319"
server_key_iv: "79491ee48e22a69a5125aa200f474f9a"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.027998 24083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "b88f20e3dd7a470c9660eff0d0e49c52"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "2d256cbbcd34650cc55941862b0ca319"
server_key_iv: "79491ee48e22a69a5125aa200f474f9a"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.032968 24083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:10:16.036345 24117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:16.037688 24083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:16.037853 24083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "b88f20e3dd7a470c9660eff0d0e49c52"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "2d256cbbcd34650cc55941862b0ca319"
server_key_iv: "79491ee48e22a69a5125aa200f474f9a"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.038043 24083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:16.067030 24083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:16.071079 24083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:16.071310 24083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:16.088342 24083 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34049
I20260809 13:10:16.088354 24206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34049 every 8 connection(s)
I20260809 13:10:16.089708 24083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:16.093310 24207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:16.098073 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 24083
I20260809 13:10:16.098271 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:10:16.098912 10966 external_mini_cluster.cc:1428] Setting key 070f4691e71e4f26ef736bac01268933
I20260809 13:10:16.103801 24207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Bootstrap starting.
I20260809 13:10:16.107368 24207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:16.108332 24207 log.cc:824] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:16.111152 24207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: No bootstrap required, opened a new log
Aug 09 13:10:16 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:16.118803 24207 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:16.119155 24207 raft_consensus.cc:374] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:16.119263 24207 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88f20e3dd7a470c9660eff0d0e49c52, State: Initialized, Role: FOLLOWER
I20260809 13:10:16.119896 24207 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:16.120153 24207 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:16.120288 24207 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:16.120467 24207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:16.120725 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:16.100844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.10.181.190:34049)
0809 13:10:16.101685 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:16.101696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:16.101739 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:10:16.103425 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:16.103451 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:16.103799 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:16.104132 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:16.106130 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.107353 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.108624 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.109689 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.109855 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:16.113583 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:16.113591 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:16.113608 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:16.113651 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:16.116589 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:16.117282 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:16.117288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:16.117294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:16.117510 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:16.117874 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:16.117881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:16.117883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:16.118403 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:16.118671 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:16.119167 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 13:10:16.119684 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":367,"threads_started":1}
I20260809 13:10:16.122803 24207 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:16.123364 24207 leader_election.cc:302] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88f20e3dd7a470c9660eff0d0e49c52; no voters:
I20260809 13:10:16.123904 24207 leader_election.cc:288] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:10:16.124004 24217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:16.124614 24217 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 LEADER]: Becoming Leader. State: Replica: b88f20e3dd7a470c9660eff0d0e49c52, State: Running, Role: LEADER
I20260809 13:10:16.125114 24217 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:16.125881 24207 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:16.127691 24221 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88f20e3dd7a470c9660eff0d0e49c52. Latest consensus state: current_term: 1 leader_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } } }
I20260809 13:10:16.128152 24221 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:16.128607 24219 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } } }
I20260809 13:10:16.128922 24219 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:16.130189 24226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:16.135219 24226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:16.144125 24226 catalog_manager.cc:1347] Generated new cluster ID: d8aea1288f754c968863a83e28cc4715
I20260809 13:10:16.144232 24226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:16.162781 24226 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:10:16.164031 24226 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:16.173887 24226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Generated new TSK 0
I20260809 13:10:16.174832 24226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:10:16.238565 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:16.569139 24252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:16.569408 24252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:16.569573 24252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:16.576462 24252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:16.576574 24252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:16.576736 24252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:16.586099 24252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:16.587288 24252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:16.588613 24252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:16.601379 24258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:16.601348 24261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:16.601418 24259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:16.606768 24252 server_base.cc:1034] running on GCE node
I20260809 13:10:16.607364 24252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:16.608268 24252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:16.609498 24252 hybrid_clock.cc:648] HybridClock initialized: now 1786281016609438 us; error 42 us; skew 500 ppm
I20260809 13:10:16.609800 24252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:16 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281016, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:16.612823 24252 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:10:16.614341 24252 webserver.cc:460] Webserver started at http://127.10.181.129:41673/ using document root <none> and password file <none>
I20260809 13:10:16.615022 24252 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:16.615150 24252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:16.615450 24252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:16.618316 24252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "54d950ed01b54bc09cb8baa03c90019d"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "516e49b95826ecd184284800a5d7696c"
server_key_iv: "b57a5a9d4d7779e88f539f3791c69f32"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.619071 24252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "54d950ed01b54bc09cb8baa03c90019d"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "516e49b95826ecd184284800a5d7696c"
server_key_iv: "b57a5a9d4d7779e88f539f3791c69f32"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.623986 24252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:10:16.627511 24276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:16.628746 24252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:10:16.628893 24252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "54d950ed01b54bc09cb8baa03c90019d"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "516e49b95826ecd184284800a5d7696c"
server_key_iv: "b57a5a9d4d7779e88f539f3791c69f32"
server_key_version: "encryptionkey@0"
I20260809 13:10:16.629094 24252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:16.651649 24252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:16.655380 24252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:16.655642 24252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:16.656530 24252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:16.657847 24252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:16.657922 24252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:16.658028 24252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:16.658082 24252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:16.689838 24252 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:46085
I20260809 13:10:16.689901 24473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:46085 every 8 connection(s)
I20260809 13:10:16.691157 24252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:16.693219 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 24252
I20260809 13:10:16.693396 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:10:16.694221 10966 external_mini_cluster.cc:1428] Setting key 7b446393720cc6fbae02622a8ffd4346
Aug 09 13:10:16 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281016, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:16.711898 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:16.693783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:43321 (local address 127.10.181.190:34049)
0809 13:10:16.694290 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:16.694296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:16.695267 (+ 971us) server_negotiation.cc:407] Connection header received
0809 13:10:16.697537 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:16.697542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:16.697606 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:16.697695 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:16.699611 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.700332 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.702394 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.702782 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.702876 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:16.706563 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:16.706568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:16.706579 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:16.706611 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:16.708946 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:16.709676 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:16.709680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:16.709682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:16.709801 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:16.710222 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:16.710226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:16.710228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:16.710489 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:16.710632 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:16.711317 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 13:10:16.711536 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":235,"threads_started":1}
I20260809 13:10:16.712766 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:16.694285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:43321)
0809 13:10:16.695081 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:16.695139 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:16.697239 (+ 2100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:16.697951 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:16.697961 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:16.698676 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:16.699454 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:16.699479 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.700506 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.700511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:16.702254 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:16.702268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:16.703003 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:16.703010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:16.703269 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:16.703930 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:16.703958 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:16.706359 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:16.709088 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:16.709097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:16.709112 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:16.709556 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:16.709908 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:16.709913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:16.709917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:16.710106 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:16.710623 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:16.710630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:16.710934 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 13:10:16.711171 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 13:10:16.711664 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":297,"threads_started":1}
I20260809 13:10:16.714293 24475 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34049
I20260809 13:10:16.714622 24475 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:16.715503 24475 heartbeater.cc:511] Master 127.10.181.190:34049 requested a full tablet report, sending...
I20260809 13:10:16.717979 24144 ts_manager.cc:194] Registered new tserver with Master: 54d950ed01b54bc09cb8baa03c90019d (127.10.181.129:46085)
I20260809 13:10:16.720005 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:43321
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:10:16.756292 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:17.083837 24490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:17.084088 24490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:17.084187 24490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:17.093475 24490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:17.093596 24490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:17.093760 24490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:17.108495 24490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:17.109943 24490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:17.111299 24490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:17.123616 24512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:17.123785 24505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:17.123646 24506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:17.124902 24490 server_base.cc:1034] running on GCE node
I20260809 13:10:17.125474 24490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:17.126433 24490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:17.127632 24490 hybrid_clock.cc:648] HybridClock initialized: now 1786281017127586 us; error 33 us; skew 500 ppm
I20260809 13:10:17.127925 24490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:17 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281017, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:17.130920 24490 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:10:17.132398 24490 webserver.cc:460] Webserver started at http://127.10.181.130:45057/ using document root <none> and password file <none>
I20260809 13:10:17.133065 24490 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:17.133196 24490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:17.133539 24490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:17.136294 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "71b278e47807499cbc3e1580a2825170"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "b32acf9553ec0ee5f457712648e493c9"
server_key_iv: "79cc893d33ca89130e372ad63c118367"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.137042 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "71b278e47807499cbc3e1580a2825170"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "b32acf9553ec0ee5f457712648e493c9"
server_key_iv: "79cc893d33ca89130e372ad63c118367"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.142059 24490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 13:10:17.145344 24524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.146560 24490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:17.146708 24490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "71b278e47807499cbc3e1580a2825170"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "b32acf9553ec0ee5f457712648e493c9"
server_key_iv: "79cc893d33ca89130e372ad63c118367"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.146883 24490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:17.173897 24490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:17.177578 24490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:17.177858 24490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:17.178995 24490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:17.180684 24490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:17.180760 24490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.180848 24490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:17.180891 24490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.210992 24490 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:35485
I20260809 13:10:17.211040 24706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:35485 every 8 connection(s)
I20260809 13:10:17.212186 24490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:17.219925 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 24490
I20260809 13:10:17.220082 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:10:17.220515 10966 external_mini_cluster.cc:1428] Setting key 9900e5bf79c624cfde7d5b0c62ceb9e3
Aug 09 13:10:17 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281017, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:17.231751 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:17.214614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:46415 (local address 127.10.181.190:34049)
0809 13:10:17.215027 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:17.215031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:17.215880 (+ 849us) server_negotiation.cc:407] Connection header received
0809 13:10:17.217686 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:17.217692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:17.217771 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:17.217868 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:17.219322 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.220054 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.221745 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.222246 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.222377 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:17.226129 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:17.226134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:17.226137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:17.226170 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:17.228665 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.229392 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.229396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.229398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.229549 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.229979 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.229983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.229985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.230242 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:17.230378 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:17.231116 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 13:10:17.231346 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":289,"thread_start_us":196,"threads_started":1}
I20260809 13:10:17.232746 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:17.215053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:46415)
0809 13:10:17.215732 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:17.215767 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:17.217389 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:17.218110 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:17.218118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:17.218609 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:17.219151 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:17.219169 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.220204 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.220208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:17.221599 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:17.221613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.222455 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.222463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:17.222758 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:17.223417 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:17.223449 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:17.225910 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:17.228800 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:17.228809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:17.228824 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:17.229264 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:17.229665 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:17.229671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:17.229675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:17.229858 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:17.230381 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:17.230389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:17.230734 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 13:10:17.230959 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 13:10:17.231510 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":253,"threads_started":1}
I20260809 13:10:17.234149 24708 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34049
I20260809 13:10:17.234431 24708 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:17.235113 24708 heartbeater.cc:511] Master 127.10.181.190:34049 requested a full tablet report, sending...
I20260809 13:10:17.236480 24144 ts_manager.cc:194] Registered new tserver with Master: 71b278e47807499cbc3e1580a2825170 (127.10.181.130:35485)
I20260809 13:10:17.237286 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:46415
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:10:17.275977 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:17.598598 24715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:17.598834 24715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:17.598932 24715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:17.605980 24715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:17.606081 24715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:17.606205 24715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:17.615487 24715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:17.616626 24715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:17.617915 24715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:17.630218 24729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:17.630424 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:17.630540 24728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:17.631470 24715 server_base.cc:1034] running on GCE node
I20260809 13:10:17.632097 24715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:17.632956 24715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:17.634155 24715 hybrid_clock.cc:648] HybridClock initialized: now 1786281017634077 us; error 62 us; skew 500 ppm
I20260809 13:10:17.634446 24715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:17 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281017, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:17.637399 24715 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:10:17.638829 24715 webserver.cc:460] Webserver started at http://127.10.181.131:43915/ using document root <none> and password file <none>
I20260809 13:10:17.639492 24715 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:17.639609 24715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:17.639917 24715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:17.642788 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "988fed8ba4a245d384de9354baa10230"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "03e31b2b4c8978a58d8203d852804586"
server_key_iv: "5e2e73d1e3915de9c2df7baa3383031e"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.643569 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "988fed8ba4a245d384de9354baa10230"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "03e31b2b4c8978a58d8203d852804586"
server_key_iv: "5e2e73d1e3915de9c2df7baa3383031e"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.648428 24715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 13:10:17.651775 24756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.652987 24715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:10:17.653151 24715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "988fed8ba4a245d384de9354baa10230"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "03e31b2b4c8978a58d8203d852804586"
server_key_iv: "5e2e73d1e3915de9c2df7baa3383031e"
server_key_version: "encryptionkey@0"
I20260809 13:10:17.653335 24715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:17.683701 24715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:17.687745 24715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:17.688006 24715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:17.688912 24715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:17.690382 24715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:17.690460 24715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.690580 24715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:17.690634 24715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:17.720878 24715 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:38913
I20260809 13:10:17.720919 24938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:38913 every 8 connection(s)
I20260809 13:10:17.722129 24715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:17.723553 24475 heartbeater.cc:503] Master 127.10.181.190:34049 was elected leader, sending a full tablet report...
I20260809 13:10:17.726333 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 24715
I20260809 13:10:17.726490 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:10:17.726945 10966 external_mini_cluster.cc:1428] Setting key 29c9310166a3528fa7a829f278aa6fac
Aug 09 13:10:17 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281017, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:17.742195 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:17.724646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:35407 (local address 127.10.181.190:34049)
0809 13:10:17.724869 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:17.724875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:17.726064 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 13:10:17.728055 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:17.728061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:17.728155 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:17.728248 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:17.730149 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.730944 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.732596 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.732990 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.733104 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:17.736400 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:17.736405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:17.736408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:17.736440 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:17.738904 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.739618 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.739623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.739625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.739741 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.740178 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.740183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.740186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.740526 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:17.740678 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:17.741642 (+ 964us) server_negotiation.cc:299] Negotiation successful
0809 13:10:17.741866 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:10:17.743010 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:17.725161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:35407)
0809 13:10:17.725880 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:17.725961 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:17.727798 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:17.728470 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:17.728478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:17.729163 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:17.729963 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:17.729990 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.731138 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.731144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:17.732454 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:17.732467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.733220 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.733228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:17.733515 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:17.734129 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:17.734157 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:17.736199 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:17.739044 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:17.739053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:17.739069 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:17.739486 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:17.739854 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:17.739859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:17.739863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:17.740040 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:17.740757 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:17.740764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:17.741161 (+ 397us) client_negotiation.cc:769] Sending connection context
0809 13:10:17.741490 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 13:10:17.741956 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":310,"threads_started":1}
I20260809 13:10:17.744150 24940 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34049
I20260809 13:10:17.744423 24940 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:17.745093 24940 heartbeater.cc:511] Master 127.10.181.190:34049 requested a full tablet report, sending...
I20260809 13:10:17.746440 24144 ts_manager.cc:194] Registered new tserver with Master: 988fed8ba4a245d384de9354baa10230 (127.10.181.131:38913)
I20260809 13:10:17.747200 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:35407
I20260809 13:10:17.753994 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
Entry for principal oryx/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab.
WARNING: no policy specified for oryx/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
Entry for principal oryx/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab.
WARNING: no policy specified for oryx/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
Entry for principal oryx/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab.
I20260809 13:10:17.897315 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 24252
Aug 09 13:10:17 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:10:17.912403 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:17.898461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.10.181.129:46085)
0809 13:10:17.899103 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:17.899111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:17.899141 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:10:17.899213 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:17.899220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:17.899457 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:17.899634 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:17.900900 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.901980 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.903183 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:17.903738 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:17.903838 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:17.907555 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:17.907568 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:17.907583 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:17.907625 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:17.910194 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.910766 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.910772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.910774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.910900 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:17.911181 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:17.911184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:17.911186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:17.911504 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:17.911737 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:17.911928 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 13:10:17.912120 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":313,"threads_started":1}
I20260809 13:10:17.913718 24468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:17.995579 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 24252
I20260809 13:10:18.016599 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 24490
Aug 09 13:10:18 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:10:18.031785 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:18.017484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.10.181.130:35485)
0809 13:10:18.018088 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:18.018096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:18.018124 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:10:18.018185 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:18.018192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:18.018397 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:18.018520 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:18.019721 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:18.020714 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:18.022057 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:18.022599 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:18.022694 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:18.026677 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:18.026690 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:18.026705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:18.026736 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:18.029329 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:18.029970 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:18.029977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:18.029980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:18.030109 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:18.030385 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:18.030388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:18.030390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:18.030769 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:18.031003 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:18.031255 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 13:10:18.031459 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":303,"threads_started":1}
I20260809 13:10:18.033077 24699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:18.113385 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 24490
I20260809 13:10:18.137612 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 24715
Aug 09 13:10:18 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:10:18.153854 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:18.138657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.10.181.131:38913)
0809 13:10:18.139326 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:18.139334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:18.139362 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:10:18.139446 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:18.139452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:18.139694 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:18.139844 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:18.141232 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:18.142382 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:18.143748 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:18.144287 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:18.144402 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:18.148480 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:18.148490 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:18.148504 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:18.148536 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:18.151229 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:18.151846 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:18.151853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:18.151855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:18.151986 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:18.152339 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:18.152343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:18.152345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:18.152686 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:18.152987 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:18.153282 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 13:10:18.153547 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":339,"threads_started":1}
I20260809 13:10:18.155089 24933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:18.237859 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 24715
I20260809 13:10:18.262250 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 24083
I20260809 13:10:18.263412 24198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:18.365658 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 24083
I20260809 13:10:18.392283 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34049
--webserver_interface=127.10.181.190
--webserver_port=38951
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:18.716760 24970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:18.717057 24970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:18.717157 24970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:18.724277 24970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:18.724377 24970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:18.724429 24970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:18.724485 24970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:18.724543 24970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:18.734050 24970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=38951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:18.735353 24970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:18.736634 24970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:18.747687 24981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:18.747987 24983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:18.748000 24988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:18.748890 24970 server_base.cc:1034] running on GCE node
I20260809 13:10:18.749643 24970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:18.750903 24970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:18.752115 24970 hybrid_clock.cc:648] HybridClock initialized: now 1786281018752056 us; error 46 us; skew 500 ppm
I20260809 13:10:18.752431 24970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:18 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281018, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:18.755350 24970 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:10:18.756784 24970 webserver.cc:460] Webserver started at http://127.10.181.190:38951/ using document root <none> and password file <none>
I20260809 13:10:18.757476 24970 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:18.757617 24970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:18.762765 24970 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:10:18.765399 25001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:18.766669 24970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:18.766817 24970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "b88f20e3dd7a470c9660eff0d0e49c52"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "2d256cbbcd34650cc55941862b0ca319"
server_key_iv: "79491ee48e22a69a5125aa200f474f9a"
server_key_version: "encryptionkey@0"
I20260809 13:10:18.767724 24970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:18.796464 24970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:18.800320 24970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:18.800715 24970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:18.818495 24970 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34049
I20260809 13:10:18.818514 25077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34049 every 8 connection(s)
I20260809 13:10:18.819973 24970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:18.826637 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 24970
I20260809 13:10:18.826972 25078 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:10:18.827675 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:46085
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:34049
--webserver_port=41673
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:10:18.831589 25078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Bootstrap starting.
I20260809 13:10:18.852779 25078 log.cc:824] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:18.861722 25078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:10:18.862231 25078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Bootstrap complete.
I20260809 13:10:18.871353 25078 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:18.872208 25078 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88f20e3dd7a470c9660eff0d0e49c52, State: Initialized, Role: FOLLOWER
I20260809 13:10:18.872833 25078 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:18.873059 25078 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:18.873171 25078 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:18.873332 25078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:10:18.876950 25078 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:18.877401 25078 leader_election.cc:302] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88f20e3dd7a470c9660eff0d0e49c52; no voters:
I20260809 13:10:18.878047 25078 leader_election.cc:288] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 13:10:18.878192 25084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:10:18.879287 25084 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [term 2 LEADER]: Becoming Leader. State: Replica: b88f20e3dd7a470c9660eff0d0e49c52, State: Running, Role: LEADER
I20260809 13:10:18.879937 25084 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } }
I20260809 13:10:18.880173 25078 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:18.882385 25085 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } } }
I20260809 13:10:18.882771 25085 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:18.884964 25086 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88f20e3dd7a470c9660eff0d0e49c52. Latest consensus state: current_term: 2 leader_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88f20e3dd7a470c9660eff0d0e49c52" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34049 } } }
I20260809 13:10:18.885278 25086 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:18.886474 25090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:18.891903 25090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:18.893293 25090 catalog_manager.cc:1259] Loaded cluster ID: d8aea1288f754c968863a83e28cc4715
I20260809 13:10:18.893468 25090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:18.897388 25090 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:18.898834 25090 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b88f20e3dd7a470c9660eff0d0e49c52: Loaded TSK: 0
I20260809 13:10:18.900362 25090 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 13:10:19.178623 25081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:19.178895 25081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:19.179025 25081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:19.187685 25081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:19.187810 25081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:19.187987 25081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:19.198182 25081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.129.keytab
--principal=oryx/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:46085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=41673
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:19.199616 25081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:19.200927 25081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:19.215677 25122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:19.215713 25124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:19.216307 25126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:19.216776 25081 server_base.cc:1034] running on GCE node
I20260809 13:10:19.217573 25081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:19.218815 25081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:19.220057 25081 hybrid_clock.cc:648] HybridClock initialized: now 1786281019219997 us; error 47 us; skew 500 ppm
I20260809 13:10:19.220417 25081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281019, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:19.224601 25081 init.cc:377] Logged in from keytab as oryx/127.10.181.129@KRBTEST.COM (short username oryx)
I20260809 13:10:19.226354 25081 webserver.cc:460] Webserver started at http://127.10.181.129:41673/ using document root <none> and password file <none>
I20260809 13:10:19.227213 25081 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:19.227358 25081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:19.234643 25081 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 13:10:19.238103 25145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.239588 25081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:10:19.239763 25081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "54d950ed01b54bc09cb8baa03c90019d"
format_stamp: "Formatted at 2026-08-09 13:10:16 on dist-test-slave-42z9"
server_key: "516e49b95826ecd184284800a5d7696c"
server_key_iv: "b57a5a9d4d7779e88f539f3791c69f32"
server_key_version: "encryptionkey@0"
I20260809 13:10:19.240715 25081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:19.269475 25081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:19.273851 25081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:19.274116 25081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:19.275110 25081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:19.276499 25081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:19.276607 25081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.276716 25081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:19.276786 25081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.306944 25081 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:46085
I20260809 13:10:19.307011 25319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:46085 every 8 connection(s)
I20260809 13:10:19.308374 25081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:19.310472 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 25081
I20260809 13:10:19.311193 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:35485
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:34049
--webserver_port=45057
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 13:10:19.328034 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.311463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53061 (local address 127.10.181.190:34049)
0809 13:10:19.312229 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.312245 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.312852 (+ 607us) server_negotiation.cc:407] Connection header received
0809 13:10:19.315080 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.315098 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.315526 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.315919 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.317263 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.319040 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.320555 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.321490 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.321661 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.327172 (+ 5511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:53061: BlockingRecv error: recv got EOF from 127.10.181.129:53061 (error 108)
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":258,"threads_started":1}
I20260809 13:10:19.328058 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.311469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:53061)
0809 13:10:19.312648 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.312694 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.314475 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.315999 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.316006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.316517 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.317065 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.317082 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.319189 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.319192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.320396 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.320408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.321687 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.321692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.321875 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.322498 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.322522 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.326868 (+ 4346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":653,"threads_started":1}
W20260809 13:10:19.328807 25320 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:34049 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.181.190:34049: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:19.337368 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.329415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:54511)
0809 13:10:19.329655 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.329675 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.329767 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.330059 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.330062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.330344 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.330505 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.330514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.331765 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.331769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.332641 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.332651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.333557 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.333562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.333689 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.334286 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.334305 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.337157 (+ 2852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 13:10:19.337677 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.329617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54511 (local address 127.10.181.190:34049)
0809 13:10:19.329794 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.329800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.329819 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:10:19.329869 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.329873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.329941 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.330030 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.330653 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.331639 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.332778 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.333352 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.333520 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.337416 (+ 3896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:54511: BlockingRecv error: recv got EOF from 127.10.181.129:54511 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:19.346030 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.338088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:53715)
0809 13:10:19.338225 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.338242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.338317 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.338726 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.338730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.339012 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.339152 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.339161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.340523 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.340527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.341496 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.341509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.342173 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.342179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.342342 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.342917 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.342936 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.345761 (+ 2825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 13:10:19.346237 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.338298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53715 (local address 127.10.181.190:34049)
0809 13:10:19.338472 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.338476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.338489 (+ 13us) server_negotiation.cc:407] Connection header received
0809 13:10:19.338528 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.338531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.338600 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.338682 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.339331 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.340391 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.341632 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.342040 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.342147 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.345979 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:53715: BlockingRecv error: recv got EOF from 127.10.181.129:53715 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 13:10:19.346325 25320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 13:10:19.650892 25326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:19.651170 25326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:19.651301 25326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:19.658188 25326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:19.658298 25326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:19.658456 25326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:19.667630 25326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.130.keytab
--principal=oryx/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:35485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=45057
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:19.668825 25326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:19.670097 25326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:19.682535 25347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:19.682623 25338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:19.682582 25339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:19.683444 25326 server_base.cc:1034] running on GCE node
I20260809 13:10:19.683975 25326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:19.684798 25326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:19.685983 25326 hybrid_clock.cc:648] HybridClock initialized: now 1786281019685930 us; error 45 us; skew 500 ppm
I20260809 13:10:19.686268 25326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281019, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:19.689225 25326 init.cc:377] Logged in from keytab as oryx/127.10.181.130@KRBTEST.COM (short username oryx)
I20260809 13:10:19.690565 25326 webserver.cc:460] Webserver started at http://127.10.181.130:45057/ using document root <none> and password file <none>
I20260809 13:10:19.691206 25326 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:19.691332 25326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:19.696767 25326 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:10:19.699401 25358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.700625 25326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:19.700778 25326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "71b278e47807499cbc3e1580a2825170"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "b32acf9553ec0ee5f457712648e493c9"
server_key_iv: "79cc893d33ca89130e372ad63c118367"
server_key_version: "encryptionkey@0"
I20260809 13:10:19.701735 25326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:19.715499 25326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:19.719369 25326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:19.719646 25326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:19.720575 25326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:19.721853 25326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:19.721953 25326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.722066 25326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:19.722137 25326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:19.751506 25326 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:35485
I20260809 13:10:19.751603 25541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:35485 every 8 connection(s)
I20260809 13:10:19.752889 25326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:19.762053 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 25326
I20260809 13:10:19.762766 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:38913
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:34049
--webserver_port=43915
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 13:10:19.769055 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.755365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:52059 (local address 127.10.181.190:34049)
0809 13:10:19.755551 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.755555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.756703 (+ 1148us) server_negotiation.cc:407] Connection header received
0809 13:10:19.758560 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.758566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.758650 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.758738 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.760219 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.760941 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.763015 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.763562 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.763666 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.768834 (+ 5168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:52059: BlockingRecv error: recv got EOF from 127.10.181.130:52059 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:10:19.769611 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.755852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:52059)
0809 13:10:19.756543 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.756594 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.758300 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.758977 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.758986 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.759501 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.760054 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.760072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.761099 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.761103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.762837 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.762854 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.763996 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.764003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.764266 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.764942 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.764968 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.768599 (+ 3631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":269,"threads_started":1}
W20260809 13:10:19.770231 25543 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:34049 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.181.190:34049: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:19.779276 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.770785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:51063)
0809 13:10:19.770957 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.770976 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.771068 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.771377 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.771382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.771672 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.771823 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.771831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.772919 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.772923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.773838 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.773850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.774647 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.774653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.774811 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.775397 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.775419 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.779020 (+ 3601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:10:19.779721 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.770862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51063 (local address 127.10.181.190:34049)
0809 13:10:19.771032 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.771038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.771057 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:10:19.771176 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.771180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.771245 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.771356 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.771928 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.772796 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.773968 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.774503 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.774610 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.779286 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:51063: BlockingRecv error: recv got EOF from 127.10.181.130:51063 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:19 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:19.792260 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:19.783402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:51343)
0809 13:10:19.783572 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:19.783596 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:19.783693 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:19.784345 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:19.784349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:19.784713 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:19.784898 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.784909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.786033 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.786038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.787383 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:19.787400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.788051 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.788058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:19.788271 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.788812 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:19.788836 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:19.791959 (+ 3123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 13:10:19.792285 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:19.783868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51343 (local address 127.10.181.190:34049)
0809 13:10:19.784035 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:19.784041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:19.784058 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:10:19.784123 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:19.784127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:19.784206 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:19.784306 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:19.785037 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.785887 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.787503 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:19.787906 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:19.788007 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:19.792130 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:51343: BlockingRecv error: recv got EOF from 127.10.181.130:51343 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 13:10:19.792654 25543 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 13:10:20.102294 25548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:20.102545 25548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:20.102684 25548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:20.109724 25548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:20.109825 25548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:20.109977 25548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:20.119323 25548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/krb5kdc/oryx_127.10.181.131.keytab
--principal=oryx/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=43915
--tserver_master_addrs=127.10.181.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:20.120575 25548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:20.121846 25548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:20.134106 25555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:20.134083 25556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:20.134157 25560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:20.134907 25548 server_base.cc:1034] running on GCE node
I20260809 13:10:20.135519 25548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:20.136365 25548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:20.137533 25548 hybrid_clock.cc:648] HybridClock initialized: now 1786281020137478 us; error 46 us; skew 500 ppm
I20260809 13:10:20.137825 25548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281020, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:20.140792 25548 init.cc:377] Logged in from keytab as oryx/127.10.181.131@KRBTEST.COM (short username oryx)
I20260809 13:10:20.142128 25548 webserver.cc:460] Webserver started at http://127.10.181.131:43915/ using document root <none> and password file <none>
I20260809 13:10:20.142778 25548 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:20.142908 25548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:20.148517 25548 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 13:10:20.151373 25569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:20.152683 25548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:10:20.152842 25548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "988fed8ba4a245d384de9354baa10230"
format_stamp: "Formatted at 2026-08-09 13:10:17 on dist-test-slave-42z9"
server_key: "03e31b2b4c8978a58d8203d852804586"
server_key_iv: "5e2e73d1e3915de9c2df7baa3383031e"
server_key_version: "encryptionkey@0"
I20260809 13:10:20.153820 25548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:20.179905 25548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:20.183593 25548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:20.183851 25548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:20.184778 25548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:20.186067 25548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:20.186163 25548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:20.186266 25548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:20.186334 25548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:20.216969 25548 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:38913
I20260809 13:10:20.217036 25744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:38913 every 8 connection(s)
I20260809 13:10:20.218469 25548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:20.225685 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 25548
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 13:10:20.234069 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:20.220823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:56399 (local address 127.10.181.190:34049)
0809 13:10:20.220986 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.220990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.222111 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 13:10:20.223906 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.223912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.223993 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.224088 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.225656 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.226548 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.228314 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.228950 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.229083 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.233795 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:56399: BlockingRecv error: recv got EOF from 127.10.181.131:56399 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:10:20.234506 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.221234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:56399)
0809 13:10:20.221947 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:20.221992 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:20.223645 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:20.224314 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:20.224322 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:20.224822 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:20.225465 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.225496 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.226710 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.226716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.228161 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.228175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.229143 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.229149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.229338 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.229972 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:20.229995 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:20.233566 (+ 3571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":283,"threads_started":1}
W20260809 13:10:20.235124 25745 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:34049 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.181.190:34049: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:20.242167 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.226654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.10.181.190:34049)
0809 13:10:20.227379 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.227385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.227401 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:10:20.227445 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.227450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.227509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.227593 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.229377 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.230179 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.231403 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.232027 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.232122 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.234953 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:20.234961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:20.234976 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:20.235012 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:20.238543 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:20.239166 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:20.239175 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:20.239184 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:20.239405 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:20.239811 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:20.239814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:20.239817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:20.240412 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:20.240720 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:20.241133 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 13:10:20.241891 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":483,"threads_started":1}
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:20.243511 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.235651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:45711)
0809 13:10:20.235834 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:20.235851 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:20.235952 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:20.236278 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:20.236283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:20.236660 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:20.236832 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.236842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.237860 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.237865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.239061 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.239074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.239907 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.239913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.240080 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.240595 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:20.240612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:20.243258 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 13:10:20.243656 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:20.235731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:45711 (local address 127.10.181.190:34049)
0809 13:10:20.235886 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.235891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.235907 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:10:20.236077 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.236081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.236142 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.236217 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.236977 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.237725 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.239213 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.239769 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.239867 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.243420 (+ 3553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:45711: BlockingRecv error: recv got EOF from 127.10.181.131:45711 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:20.250908 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.244343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:46073)
0809 13:10:20.244509 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:20.244528 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:20.244624 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:20.244894 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:20.244897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:20.245152 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:20.245356 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.245369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.246451 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.246455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.247319 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.247330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.247892 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.247896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.248008 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.248429 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:20.248444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:20.250711 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:10:20.251060 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:20.244424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:46073 (local address 127.10.181.190:34049)
0809 13:10:20.244657 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.244662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.244680 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:10:20.244722 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.244727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.244790 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.244866 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.245525 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.246333 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.247422 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.247797 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.247880 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.250873 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:46073: BlockingRecv error: recv got EOF from 127.10.181.131:46073 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
W20260809 13:10:20.251178 25745 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:20.357167 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.346967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:42095)
0809 13:10:20.347502 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:20.347521 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:20.347692 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:20.348126 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:20.348130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:20.348640 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:20.348843 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.348852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.350176 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.350181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.351567 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.351585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.352395 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.352400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.352548 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.353203 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:20.353222 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:20.356841 (+ 3619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":252,"threads_started":1}
W20260809 13:10:20.357426 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:20.347024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:42095 (local address 127.10.181.190:34049)
0809 13:10:20.347249 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.347256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.347595 (+ 339us) server_negotiation.cc:407] Connection header received
0809 13:10:20.347809 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.347813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.347878 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.347932 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.348982 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.349961 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.351793 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.352233 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.352335 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.357084 (+ 4749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:42095: BlockingRecv error: recv got EOF from 127.10.181.129:42095 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:20 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:20.801195 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:20.793266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:54179)
0809 13:10:20.793869 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:20.793890 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:20.794020 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:20.794366 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:20.794371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:20.794843 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:20.795051 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.795061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.795948 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.795951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.797010 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:20.797024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.797716 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.797720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:20.797846 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.798364 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:20.798381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:20.800950 (+ 2569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":340,"threads_started":1}
W20260809 13:10:20.801359 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:20.793298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:54179 (local address 127.10.181.190:34049)
0809 13:10:20.793655 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:20.793667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:20.793967 (+ 300us) server_negotiation.cc:407] Connection header received
0809 13:10:20.794139 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:20.794145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:20.794213 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:20.794310 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:20.795168 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.795843 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.797143 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:20.797572 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:20.797658 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:20.801151 (+ 3493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:54179: BlockingRecv error: recv got EOF from 127.10.181.130:54179 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:21.265643 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:21.251701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:60763)
0809 13:10:21.252186 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:21.252201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:21.252333 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:21.252662 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:21.252667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:21.253144 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:21.253347 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.253357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.254390 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.254396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.255435 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.255449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.256156 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.256163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.256326 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.262709 (+ 6383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:21.262728 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:21.265305 (+ 2577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
W20260809 13:10:21.265789 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:21.251773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60763 (local address 127.10.181.190:34049)
0809 13:10:21.251954 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:21.251961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:21.252288 (+ 327us) server_negotiation.cc:407] Connection header received
0809 13:10:21.252450 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:21.252454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:21.252517 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:21.252596 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:21.253515 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.254241 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.255577 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.255973 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.256082 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.265569 (+ 9487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:60763: BlockingRecv error: recv got EOF from 127.10.181.131:60763 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:21.366459 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:21.358183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:57773)
0809 13:10:21.358733 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:21.358749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:21.358919 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:21.359245 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:21.359249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:21.359685 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:21.359885 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.359895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.360906 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.360911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.361850 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.361863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.362581 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.362587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.362761 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.363361 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:21.363426 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:21.366230 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":314,"threads_started":1}
W20260809 13:10:21.366641 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:21.358263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:57773 (local address 127.10.181.190:34049)
0809 13:10:21.358472 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:21.358479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:21.358833 (+ 354us) server_negotiation.cc:407] Connection header received
0809 13:10:21.359041 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:21.359045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:21.359109 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:21.359188 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:21.360026 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.360759 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.361990 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.362422 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.362540 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.366411 (+ 3871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:57773: BlockingRecv error: recv got EOF from 127.10.181.129:57773 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:21 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:21.810528 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:21.802172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:35425)
0809 13:10:21.802724 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:21.802739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:21.802859 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:21.803221 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:21.803226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:21.803706 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:21.803885 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.803895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.804953 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.804956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.805983 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:21.805996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.806738 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.806742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:21.806855 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.807361 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:21.807377 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:21.810189 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":304,"threads_started":1}
W20260809 13:10:21.810631 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:21.802277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:35425 (local address 127.10.181.190:34049)
0809 13:10:21.802489 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:21.802495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:21.802828 (+ 333us) server_negotiation.cc:407] Connection header received
0809 13:10:21.802970 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:21.802975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:21.803059 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:21.803167 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:21.804083 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.804832 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.806120 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:21.806604 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:21.806695 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:21.810407 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:35425: BlockingRecv error: recv got EOF from 127.10.181.130:35425 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:22.275070 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:22.266573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:40947)
0809 13:10:22.267047 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:22.267063 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:22.267181 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:22.267462 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:22.267465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:22.267896 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:22.268073 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.268084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.269168 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.269174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.270482 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.270497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.271291 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.271296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.271449 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.272068 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:22.272086 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:22.274838 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":255,"threads_started":1}
W20260809 13:10:22.275372 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:22.266737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:40947 (local address 127.10.181.190:34049)
0809 13:10:22.266959 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:22.266965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:22.267136 (+ 171us) server_negotiation.cc:407] Connection header received
0809 13:10:22.267272 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:22.267275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:22.267342 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:22.267420 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:22.268180 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.269021 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.270618 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.271106 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.271244 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.275105 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:40947: BlockingRecv error: recv got EOF from 127.10.181.131:40947 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:22.375099 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:22.367275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:45879)
0809 13:10:22.367742 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:22.367757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:22.367868 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:22.368134 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:22.368137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:22.368479 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:22.368662 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.368671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.369658 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.369662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.370557 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.370569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.371141 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.371145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.371260 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.371845 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:22.371865 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:22.374828 (+ 2963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":224,"threads_started":1}
W20260809 13:10:22.375236 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:22.367339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:45879 (local address 127.10.181.190:34049)
0809 13:10:22.367580 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:22.367586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:22.367829 (+ 243us) server_negotiation.cc:407] Connection header received
0809 13:10:22.367960 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:22.367963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:22.368026 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:22.368102 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:22.368771 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.369558 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.370665 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.371034 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.371128 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.375009 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:45879: BlockingRecv error: recv got EOF from 127.10.181.129:45879 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:22 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:22.820149 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:22.811452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:59637)
0809 13:10:22.812083 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:22.812102 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:22.812252 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:22.812587 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:22.812592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:22.812967 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:22.813163 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.813173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.814310 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.814315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.815344 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:22.815360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.816041 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.816047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:22.816212 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.816755 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:22.816772 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:22.819803 (+ 3031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":353,"threads_started":1}
W20260809 13:10:22.820205 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:22.811543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59637 (local address 127.10.181.190:34049)
0809 13:10:22.811724 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:22.811730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:22.812188 (+ 458us) server_negotiation.cc:407] Connection header received
0809 13:10:22.812380 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:22.812385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:22.812450 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:22.812530 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:22.813317 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.814163 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.815488 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:22.815896 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:22.816004 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:22.819989 (+ 3985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:59637: BlockingRecv error: recv got EOF from 127.10.181.130:59637 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:23.283984 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:23.275966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:56251)
0809 13:10:23.276501 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:23.276517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:23.276632 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:23.276916 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:23.276919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:23.277300 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:23.277517 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.277529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.278821 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.278825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.279729 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.279744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.280368 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.280373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.280504 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.281040 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:23.281058 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:23.283732 (+ 2674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":270,"threads_started":1}
W20260809 13:10:23.284083 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:23.276045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:56251 (local address 127.10.181.190:34049)
0809 13:10:23.276211 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:23.276215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:23.276589 (+ 374us) server_negotiation.cc:407] Connection header received
0809 13:10:23.276748 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:23.276751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:23.276813 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:23.276887 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:23.277671 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.278687 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.279837 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.280228 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.280335 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.283907 (+ 3572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:56251: BlockingRecv error: recv got EOF from 127.10.181.131:56251 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:23.384121 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:23.375957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:34181)
0809 13:10:23.376447 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:23.376462 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:23.376593 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:23.376921 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:23.376926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:23.377341 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:23.377543 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.377555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.378545 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.378550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.379526 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.379539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.380235 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.380242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.380404 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.380988 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:23.381014 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:23.383890 (+ 2876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":270,"threads_started":1}
W20260809 13:10:23.384295 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:23.376015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:34181 (local address 127.10.181.190:34049)
0809 13:10:23.376237 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:23.376244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:23.376548 (+ 304us) server_negotiation.cc:407] Connection header received
0809 13:10:23.376713 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:23.376719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:23.376790 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:23.376870 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:23.377682 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.378412 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.379663 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.380096 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.380205 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.384079 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:34181: BlockingRecv error: recv got EOF from 127.10.181.129:34181 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:23 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:23.829373 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:23.821147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:34967)
0809 13:10:23.821819 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:23.821835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:23.821948 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:23.822255 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:23.822258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:23.822629 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:23.822788 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.822797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.824207 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.824211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.825181 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:23.825194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.825892 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.825895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:23.826016 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.826525 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:23.826542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:23.829156 (+ 2614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":459,"threads_started":1}
W20260809 13:10:23.829552 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:23.821200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:34967 (local address 127.10.181.190:34049)
0809 13:10:23.821413 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:23.821421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:23.821917 (+ 496us) server_negotiation.cc:407] Connection header received
0809 13:10:23.822046 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:23.822052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:23.822131 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:23.822227 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:23.822924 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.824075 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.825322 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:23.825784 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:23.825881 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:23.829329 (+ 3448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:34967: BlockingRecv error: recv got EOF from 127.10.181.130:34967 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:24.292805 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:24.284824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:48981)
0809 13:10:24.285338 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:24.285355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:24.285493 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:24.285787 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:24.285790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:24.286165 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:24.286319 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.286328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.287380 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.287384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.288405 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.288419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.289035 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.289038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.289157 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.289774 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:24.289793 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:24.292475 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
W20260809 13:10:24.292932 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:24.284881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:48981 (local address 127.10.181.190:34049)
0809 13:10:24.285048 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:24.285052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:24.285458 (+ 406us) server_negotiation.cc:407] Connection header received
0809 13:10:24.285598 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:24.285602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:24.285667 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:24.285744 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:24.286427 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.287267 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.288511 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.288925 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.289022 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.292698 (+ 3676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:48981: BlockingRecv error: recv got EOF from 127.10.181.131:48981 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:24.393410 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:24.385048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:50983)
0809 13:10:24.385659 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:24.385678 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:24.385822 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:24.386175 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:24.386180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:24.386592 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:24.386778 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.386789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.387907 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.387912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.388943 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.388957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.389733 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.389739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.389909 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.390478 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:24.390497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:24.393162 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":242,"threads_started":1}
W20260809 13:10:24.393555 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:24.385109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:50983 (local address 127.10.181.190:34049)
0809 13:10:24.385323 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:24.385330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:24.385794 (+ 464us) server_negotiation.cc:407] Connection header received
0809 13:10:24.385949 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:24.385968 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:24.386034 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:24.386124 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:24.386938 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.387754 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.389101 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.389584 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.389709 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.393339 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:50983: BlockingRecv error: recv got EOF from 127.10.181.129:50983 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:24 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:24.838299 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:24.830228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:51015)
0809 13:10:24.830720 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:24.830736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:24.830859 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:24.831163 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:24.831168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:24.831588 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:24.831759 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.831770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.832757 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.832763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.833792 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:24.833806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.834499 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.834506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:24.834669 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.835174 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:24.835191 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:24.838050 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":210,"threads_started":1}
W20260809 13:10:24.838419 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:24.830412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51015 (local address 127.10.181.190:34049)
0809 13:10:24.830631 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:24.830638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:24.830804 (+ 166us) server_negotiation.cc:407] Connection header received
0809 13:10:24.830965 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:24.830971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:24.831040 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:24.831117 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:24.831895 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.832623 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.833925 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:24.834358 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:24.834464 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:24.838214 (+ 3750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:51015: BlockingRecv error: recv got EOF from 127.10.181.130:51015 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:25.302132 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:25.293938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:49869)
0809 13:10:25.294469 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:25.294483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:25.294614 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:25.294925 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:25.294931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:25.295347 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:25.295513 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.295522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.296501 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.296506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.297463 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.297480 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.298172 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.298178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.298342 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.298938 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:25.298956 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:25.301877 (+ 2921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":257,"threads_started":1}
W20260809 13:10:25.302286 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:25.294013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:49869 (local address 127.10.181.190:34049)
0809 13:10:25.294211 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:25.294217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:25.294568 (+ 351us) server_negotiation.cc:407] Connection header received
0809 13:10:25.294727 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:25.294732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:25.294798 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:25.294879 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:25.295666 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.296366 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.297614 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.298033 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.298141 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.302079 (+ 3938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:49869: BlockingRecv error: recv got EOF from 127.10.181.131:49869 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:25.402133 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:25.394424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:47559)
0809 13:10:25.394999 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:25.395017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:25.395138 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:25.395422 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:25.395425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:25.395816 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:25.395987 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.395996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.396952 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.396955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.397868 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.397889 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.398476 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.398480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.398602 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.399185 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:25.399204 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:25.401910 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":307,"threads_started":1}
W20260809 13:10:25.402314 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:25.394467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:47559 (local address 127.10.181.190:34049)
0809 13:10:25.394626 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:25.394630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:25.395087 (+ 457us) server_negotiation.cc:407] Connection header received
0809 13:10:25.395246 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:25.395250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:25.395311 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:25.395389 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:25.396104 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.396840 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.397980 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.398362 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.398447 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.402092 (+ 3645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:47559: BlockingRecv error: recv got EOF from 127.10.181.129:47559 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:25 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:25.847239 25778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:25.839198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:59579)
0809 13:10:25.839752 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:25.839769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:25.839888 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:25.840167 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:25.840170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:25.840587 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:25.840755 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.840764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.841730 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.841734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.842620 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:25.842632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.843241 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.843245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:25.843362 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.844023 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:25.844044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:25.846937 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":298,"threads_started":1}
W20260809 13:10:25.847452 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:25.839262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59579 (local address 127.10.181.190:34049)
0809 13:10:25.839443 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:25.839447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:25.839836 (+ 389us) server_negotiation.cc:407] Connection header received
0809 13:10:25.839975 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:25.839979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:25.840040 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:25.840125 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:25.840870 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.841628 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.842731 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:25.843131 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:25.843213 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:25.847220 (+ 4007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:59579: BlockingRecv error: recv got EOF from 127.10.181.130:59579 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:26.314502 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:26.303001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:48269)
0809 13:10:26.303519 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:26.303534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:26.303679 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:26.303994 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:26.303999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:26.304422 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:26.304597 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.304634 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.305875 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.305881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.307176 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.307193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.307920 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.307926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.308101 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.311552 (+ 3451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:26.311570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:26.314271 (+ 2701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
W20260809 13:10:26.314689 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:26.303088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:48269 (local address 127.10.181.190:34049)
0809 13:10:26.303288 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:26.303295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:26.303611 (+ 316us) server_negotiation.cc:407] Connection header received
0809 13:10:26.303791 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:26.303796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:26.303863 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:26.303944 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:26.304756 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.305748 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.307317 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.307760 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.307863 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.314455 (+ 6592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:48269: BlockingRecv error: recv got EOF from 127.10.181.131:48269 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:26.410962 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:26.402971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:56217)
0809 13:10:26.403519 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:26.403536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:26.403663 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:26.403931 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:26.403934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:26.404354 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:26.404529 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.404538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.405609 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.405614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.406568 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.406582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.407222 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.407226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.407332 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.407838 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:26.407855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:26.410724 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":234,"threads_started":1}
W20260809 13:10:26.411101 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:26.403043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:56217 (local address 127.10.181.190:34049)
0809 13:10:26.403276 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:26.403284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:26.403618 (+ 334us) server_negotiation.cc:407] Connection header received
0809 13:10:26.403762 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:26.403765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:26.403827 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:26.403902 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:26.404714 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.405466 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.406707 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.407113 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.407201 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.410906 (+ 3705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:56217: BlockingRecv error: recv got EOF from 127.10.181.129:56217 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:26 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:26.856424 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:26.848313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:57195)
0809 13:10:26.848875 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:26.848891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:26.849032 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:26.849392 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:26.849398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:26.849857 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:26.850039 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.850049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.851065 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.851069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.851983 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:26.851996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.852709 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.852715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:26.852871 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.853470 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:26.853490 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:26.856163 (+ 2673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":242,"threads_started":1}
W20260809 13:10:26.856535 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:26.848341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:57195 (local address 127.10.181.190:34049)
0809 13:10:26.848567 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:26.848584 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:26.848976 (+ 392us) server_negotiation.cc:407] Connection header received
0809 13:10:26.849178 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:26.849184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:26.849254 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:26.849336 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:26.850189 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.850931 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.852122 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:26.852554 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:26.852672 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:26.856327 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:57195: BlockingRecv error: recv got EOF from 127.10.181.130:57195 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:27.323509 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:27.315392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:56055)
0809 13:10:27.315997 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:27.316013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:27.316140 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:27.316418 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:27.316421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:27.316814 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:27.316984 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.316993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.317921 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.317925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.318831 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.318843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.319494 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.319498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.319609 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.320192 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:27.320211 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:27.323250 (+ 3039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":255,"threads_started":1}
W20260809 13:10:27.323601 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:27.315458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:56055 (local address 127.10.181.190:34049)
0809 13:10:27.315619 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:27.315626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:27.316077 (+ 451us) server_negotiation.cc:407] Connection header received
0809 13:10:27.316229 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:27.316233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:27.316295 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:27.316373 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:27.317089 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.317823 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.318939 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.319366 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.319472 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.323424 (+ 3952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:56055: BlockingRecv error: recv got EOF from 127.10.181.131:56055 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:27.419914 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:27.411843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:48315)
0809 13:10:27.412308 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:27.412323 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:27.412439 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:27.412748 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:27.412751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:27.413150 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:27.413306 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.413315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.414455 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.414459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.415578 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.415593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.416301 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.416305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.416420 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.416976 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:27.416993 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:27.419684 (+ 2691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":254,"threads_started":1}
W20260809 13:10:27.420121 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:27.411992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:48315 (local address 127.10.181.190:34049)
0809 13:10:27.412208 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:27.412214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:27.412392 (+ 178us) server_negotiation.cc:407] Connection header received
0809 13:10:27.412561 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:27.412565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:27.412641 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:27.412719 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:27.413414 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.414296 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.415714 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.416159 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.416258 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.419876 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:48315: BlockingRecv error: recv got EOF from 127.10.181.129:48315 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:27 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:27.865754 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:27.857320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:57503)
0809 13:10:27.857840 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:27.857856 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:27.858007 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:27.858330 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:27.858335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:27.858761 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:27.858920 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.858929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.860128 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.860132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.861052 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:27.861066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.861862 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.861868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:27.862007 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.862592 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:27.862616 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:27.865396 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":262,"threads_started":1}
W20260809 13:10:27.865852 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:27.857402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:57503 (local address 127.10.181.190:34049)
0809 13:10:27.857574 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:27.857579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:27.857958 (+ 379us) server_negotiation.cc:407] Connection header received
0809 13:10:27.858120 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:27.858126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:27.858193 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:27.858273 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:27.859064 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.859973 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.861202 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:27.861683 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:27.861794 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:27.865643 (+ 3849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:57503: BlockingRecv error: recv got EOF from 127.10.181.130:57503 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:28.332366 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:28.324452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:57035)
0809 13:10:28.325029 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:28.325046 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:28.325190 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:28.325570 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:28.325575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:28.326017 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:28.326181 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.326190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.327198 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.327202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.328131 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.328142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.328761 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.328765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.328873 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.329411 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:28.329428 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:28.332141 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":273,"threads_started":1}
W20260809 13:10:28.332542 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:28.324528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:57035 (local address 127.10.181.190:34049)
0809 13:10:28.324725 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:28.324732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:28.325138 (+ 406us) server_negotiation.cc:407] Connection header received
0809 13:10:28.325308 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:28.325313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:28.325397 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:28.325534 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:28.326311 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.327051 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.328239 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.328629 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.328713 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.332325 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:57035: BlockingRecv error: recv got EOF from 127.10.181.131:57035 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:28.429011 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:28.420751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:49181)
0809 13:10:28.421236 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:28.421251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:28.421385 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:28.421793 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:28.421798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:28.422201 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:28.422374 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.422383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.423425 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.423429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.424364 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.424376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.425078 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.425084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.425264 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.425829 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:28.425848 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:28.428704 (+ 2856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":254,"threads_started":1}
W20260809 13:10:28.429165 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:28.420804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:49181 (local address 127.10.181.190:34049)
0809 13:10:28.420992 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:28.420998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:28.421336 (+ 338us) server_negotiation.cc:407] Connection header received
0809 13:10:28.421531 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:28.421537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:28.421643 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:28.421727 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:28.422512 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.423291 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.424499 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.424931 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.425038 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.428911 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:49181: BlockingRecv error: recv got EOF from 127.10.181.129:49181 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:28 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:28.874671 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:28.866714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:55105)
0809 13:10:28.867224 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:28.867240 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:28.867357 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:28.867629 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:28.867632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:28.868013 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:28.868208 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.868219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.869149 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.869152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.870124 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:28.870145 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.870802 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.870806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:28.870932 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.871502 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:28.871522 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:28.874337 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":264,"threads_started":1}
W20260809 13:10:28.874778 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:28.866754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:55105 (local address 127.10.181.190:34049)
0809 13:10:28.866917 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:28.866923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:28.867304 (+ 381us) server_negotiation.cc:407] Connection header received
0809 13:10:28.867444 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:28.867447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:28.867507 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:28.867588 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:28.868318 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.869043 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.870244 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:28.870658 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:28.870789 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:28.874543 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:55105: BlockingRecv error: recv got EOF from 127.10.181.130:55105 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:29.341825 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:29.333316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:51399)
0809 13:10:29.333854 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:29.333871 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:29.333987 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:29.334304 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:29.334307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:29.334706 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:29.334883 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.334892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.335854 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.335857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.336802 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.336813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.337566 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.337572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.337702 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.338279 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:29.338299 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:29.341495 (+ 3196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":315,"threads_started":1}
W20260809 13:10:29.341918 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:29.333364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:51399 (local address 127.10.181.190:34049)
0809 13:10:29.333574 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:29.333582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:29.333964 (+ 382us) server_negotiation.cc:407] Connection header received
0809 13:10:29.334079 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:29.334082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:29.334160 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:29.334264 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:29.335025 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.335760 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.336942 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.337343 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.337468 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.341696 (+ 4228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:51399: BlockingRecv error: recv got EOF from 127.10.181.131:51399 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:29.438584 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:29.429987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:54609)
0809 13:10:29.430507 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:29.430526 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:29.430653 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:29.431020 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:29.431024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:29.431513 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:29.431743 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.431760 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.432950 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.432955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.434011 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.434025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.434696 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.434700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.434812 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.435341 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:29.435358 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:29.438233 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":283,"threads_started":1}
W20260809 13:10:29.438707 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:29.430062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54609 (local address 127.10.181.190:34049)
0809 13:10:29.430219 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:29.430223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:29.430593 (+ 370us) server_negotiation.cc:407] Connection header received
0809 13:10:29.430766 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:29.430771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:29.430868 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:29.430967 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:29.431887 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.432827 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.434133 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.434583 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.434675 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.438461 (+ 3786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:54609: BlockingRecv error: recv got EOF from 127.10.181.129:54609 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:29 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:29.883296 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:29.875626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:46655)
0809 13:10:29.876108 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:29.876129 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:29.876248 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:29.876525 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:29.876528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:29.876904 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:29.877080 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.877090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.878077 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.878081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.879021 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:29.879035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.879639 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.879643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:29.879757 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.880324 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:29.880342 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:29.883072 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":245,"threads_started":1}
W20260809 13:10:29.883489 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:29.875728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:46655 (local address 127.10.181.190:34049)
0809 13:10:29.875957 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:29.875963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:29.876215 (+ 252us) server_negotiation.cc:407] Connection header received
0809 13:10:29.876346 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:29.876349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:29.876414 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:29.876484 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:29.877195 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.877938 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.879133 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:29.879533 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:29.879627 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:29.883262 (+ 3635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:46655: BlockingRecv error: recv got EOF from 127.10.181.130:46655 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:30.354094 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:30.342838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:45157)
0809 13:10:30.343365 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:30.343382 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:30.343496 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:30.343793 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:30.343796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:30.344272 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:30.344436 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.344447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.345592 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.345596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.346528 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.346541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.347157 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.347161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.347291 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.350961 (+ 3670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:30.350980 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:30.353863 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":285,"threads_started":1}
W20260809 13:10:30.354281 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:30.342975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:45157 (local address 127.10.181.190:34049)
0809 13:10:30.343200 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:30.343204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:30.343446 (+ 242us) server_negotiation.cc:407] Connection header received
0809 13:10:30.343583 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:30.343587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:30.343683 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:30.343765 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:30.344574 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.345433 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.346639 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.347048 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.347149 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.354040 (+ 6891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:45157: BlockingRecv error: recv got EOF from 127.10.181.131:45157 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:30.447753 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:30.439464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:44689)
0809 13:10:30.439970 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:30.439989 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:30.440118 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:30.440404 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:30.440407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:30.440797 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:30.440971 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.440980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.442029 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.442033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.443193 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.443207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.444020 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.444026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.444162 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.444781 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:30.444801 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:30.447503 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
W20260809 13:10:30.447916 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:30.439544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:44689 (local address 127.10.181.190:34049)
0809 13:10:30.439723 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:30.439728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:30.440054 (+ 326us) server_negotiation.cc:407] Connection header received
0809 13:10:30.440210 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:30.440213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:30.440293 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:30.440377 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:30.441085 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.441893 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.443329 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.443884 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.443979 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.447702 (+ 3723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:44689: BlockingRecv error: recv got EOF from 127.10.181.129:44689 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:30 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:30.891952 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:30.884152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:56455)
0809 13:10:30.884691 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:30.884707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:30.884824 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:30.885145 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:30.885149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:30.885591 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:30.885776 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.885786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.886741 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.886744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.887670 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:30.887683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.888322 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.888326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:30.888452 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.889026 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:30.889044 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:30.891713 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":284,"threads_started":1}
W20260809 13:10:30.892131 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:30.884234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:56455 (local address 127.10.181.190:34049)
0809 13:10:30.884462 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:30.884469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:30.884785 (+ 316us) server_negotiation.cc:407] Connection header received
0809 13:10:30.884915 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:30.884919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:30.884984 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:30.885061 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:30.885895 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.886629 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.887786 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:30.888198 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:30.888292 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:30.891921 (+ 3629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:56455: BlockingRecv error: recv got EOF from 127.10.181.130:56455 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:31.367959 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:31.354923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:33911)
0809 13:10:31.355393 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:31.355409 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:31.355540 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:31.355874 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:31.355878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:31.356306 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:31.356485 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.356495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.357541 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.357547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.358523 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.358536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.359291 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.359297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.359480 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.364864 (+ 5384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:31.364883 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:31.367607 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":237,"threads_started":1}
W20260809 13:10:31.368050 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:31.354998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:33911 (local address 127.10.181.190:34049)
0809 13:10:31.355221 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:31.355227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:31.355495 (+ 268us) server_negotiation.cc:407] Connection header received
0809 13:10:31.355667 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:31.355672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:31.355743 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:31.355823 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:31.356642 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.357400 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.358669 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.359138 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.359244 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.367818 (+ 8574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:33911: BlockingRecv error: recv got EOF from 127.10.181.131:33911 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:31.456342 25808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:31.448604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:38535)
0809 13:10:31.449210 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:31.449230 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:31.449368 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:31.449753 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:31.449757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:31.450157 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:31.450315 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.450324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.451269 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.451273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.452191 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.452203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.452857 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.452861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.452980 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.453522 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:31.453540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:31.456108 (+ 2568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":363,"threads_started":1}
W20260809 13:10:31.456518 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:31.448670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:38535 (local address 127.10.181.190:34049)
0809 13:10:31.448873 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:31.448879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:31.449310 (+ 431us) server_negotiation.cc:407] Connection header received
0809 13:10:31.449504 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:31.449508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:31.449592 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:31.449696 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:31.450430 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.451145 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.452304 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.452749 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.452836 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.456310 (+ 3474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:38535: BlockingRecv error: recv got EOF from 127.10.181.129:38535 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:31 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:31.900990 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:31.892800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:39809)
0809 13:10:31.893285 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:31.893303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:31.893414 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:31.893714 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:31.893717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:31.894127 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:31.894289 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.894298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.895192 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.895196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.896109 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:31.896120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.896756 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.896760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:31.896880 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.897556 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:31.897575 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:31.900749 (+ 3174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":277,"threads_started":1}
W20260809 13:10:31.901185 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:31.892864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:39809 (local address 127.10.181.190:34049)
0809 13:10:31.893108 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:31.893114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:31.893376 (+ 262us) server_negotiation.cc:407] Connection header received
0809 13:10:31.893524 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:31.893527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:31.893590 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:31.893674 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:31.894397 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.895073 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.896221 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:31.896652 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:31.896742 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:31.900969 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:39809: BlockingRecv error: recv got EOF from 127.10.181.130:39809 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:32.378341 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:32.369069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:36531)
0809 13:10:32.369603 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:32.369620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:32.369734 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:32.370063 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:32.370066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:32.370505 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:32.370690 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.370700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.372162 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.372166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.373178 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.373191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.374161 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.374165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.374300 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.374877 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:32.374897 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:32.378029 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":266,"threads_started":1}
W20260809 13:10:32.378635 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:32.369171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:36531 (local address 127.10.181.190:34049)
0809 13:10:32.369367 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:32.369373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:32.369691 (+ 318us) server_negotiation.cc:407] Connection header received
0809 13:10:32.369835 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:32.369841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:32.369925 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:32.370034 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:32.370829 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.372008 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.373338 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.374024 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.374158 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.378378 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:36531: BlockingRecv error: recv got EOF from 127.10.181.131:36531 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:32.465723 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:32.457254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:59377)
0809 13:10:32.457824 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:32.457841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:32.457953 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:32.458297 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:32.458302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:32.458870 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:32.459104 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.459119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.460314 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.460318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.461220 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.461232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.461895 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.461899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.462013 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.462603 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:32.462623 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:32.465358 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":293,"threads_started":1}
W20260809 13:10:32.465854 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:32.457375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:59377 (local address 127.10.181.190:34049)
0809 13:10:32.457605 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:32.457612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:32.457906 (+ 294us) server_negotiation.cc:407] Connection header received
0809 13:10:32.458045 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:32.458049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:32.458142 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:32.458224 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:32.459274 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.460184 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.461330 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.461763 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.461853 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.465604 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:59377: BlockingRecv error: recv got EOF from 127.10.181.129:59377 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:32 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:32.909869 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:32.901907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:48319)
0809 13:10:32.902396 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:32.902412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:32.902526 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:32.902802 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:32.902806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:32.903175 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:32.903344 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.903353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.904276 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.904279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.905224 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:32.905237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.906006 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.906011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:32.906130 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.906694 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:32.906713 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:32.909516 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":249,"threads_started":1}
W20260809 13:10:32.909960 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:32.902034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:48319 (local address 127.10.181.190:34049)
0809 13:10:32.902256 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:32.902262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:32.902482 (+ 220us) server_negotiation.cc:407] Connection header received
0809 13:10:32.902626 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:32.902630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:32.902693 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:32.902772 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:32.903449 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.904174 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.905352 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:32.905837 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:32.905954 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:32.909708 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:48319: BlockingRecv error: recv got EOF from 127.10.181.130:48319 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:33.387494 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:33.379341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:42579)
0809 13:10:33.379868 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:33.379888 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:33.380024 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:33.380329 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:33.380335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:33.380775 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:33.380953 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.380966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.382004 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.382010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.382932 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.382944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.383680 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.383686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.383836 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.384348 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:33.384366 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:33.387242 (+ 2876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":258,"threads_started":1}
W20260809 13:10:33.387615 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:33.379400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:42579 (local address 127.10.181.190:34049)
0809 13:10:33.379578 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:33.379585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:33.379964 (+ 379us) server_negotiation.cc:407] Connection header received
0809 13:10:33.380136 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:33.380140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:33.380203 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:33.380283 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:33.381097 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.381868 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.383079 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.383496 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.383603 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.387435 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:42579: BlockingRecv error: recv got EOF from 127.10.181.131:42579 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:33.474642 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:33.466644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:58573)
0809 13:10:33.467163 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:33.467183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:33.467330 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:33.467745 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:33.467750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:33.468256 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:33.468450 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.468460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.469476 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.469479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.470360 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.470370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.471015 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.471020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.471152 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.471706 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:33.471726 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:33.474388 (+ 2662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":255,"threads_started":1}
W20260809 13:10:33.474817 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:33.466706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:58573 (local address 127.10.181.190:34049)
0809 13:10:33.467001 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:33.467008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:33.467268 (+ 260us) server_negotiation.cc:407] Connection header received
0809 13:10:33.467473 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:33.467478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:33.467580 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:33.467688 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:33.468579 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.469346 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.470473 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.470889 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.471012 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.474583 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:58573: BlockingRecv error: recv got EOF from 127.10.181.129:58573 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:33 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:33.918604 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:33.910958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:51159)
0809 13:10:33.911414 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:33.911429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:33.911543 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:33.911804 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:33.911807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:33.912199 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:33.912387 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.912396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.913363 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.913367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.914315 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:33.914328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.914927 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.914930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:33.915041 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.915594 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:33.915612 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:33.918341 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":232,"threads_started":1}
W20260809 13:10:33.918728 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:33.911007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51159 (local address 127.10.181.190:34049)
0809 13:10:33.911184 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:33.911188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:33.911493 (+ 305us) server_negotiation.cc:407] Connection header received
0809 13:10:33.911634 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:33.911637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:33.911695 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:33.911775 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:33.912499 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.913227 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.914421 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:33.914822 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:33.914911 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:33.918537 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:51159: BlockingRecv error: recv got EOF from 127.10.181.130:51159 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.131@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:34.396107 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:34.388442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.131:43851)
0809 13:10:34.388929 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:34.388946 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:34.389064 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:34.389328 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:34.389331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:34.389749 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:34.389933 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.389943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.390894 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.390897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.391809 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.391820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.392437 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.392441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.392554 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.393174 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:34.393192 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:34.395826 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
W20260809 13:10:34.396332 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:34.388508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:43851 (local address 127.10.181.190:34049)
0809 13:10:34.388682 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:34.388687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:34.389010 (+ 323us) server_negotiation.cc:407] Connection header received
0809 13:10:34.389154 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:34.389158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:34.389220 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:34.389297 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:34.390050 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.390775 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.391915 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.392336 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.392423 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.396094 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.131:43851: BlockingRecv error: recv got EOF from 127.10.181.131:43851 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.129@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:34.483928 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:34.475712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.129:52457)
0809 13:10:34.476257 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:34.476273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:34.476385 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:34.476682 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:34.476685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:34.477072 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:34.477251 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.477260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.478633 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.478639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.479660 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.479673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.480390 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.480394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.480507 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.481034 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:34.481051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:34.483681 (+ 2630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":297,"threads_started":1}
W20260809 13:10:34.484086 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:34.475807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:52457 (local address 127.10.181.190:34049)
0809 13:10:34.475964 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:34.475969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:34.476336 (+ 367us) server_negotiation.cc:407] Connection header received
0809 13:10:34.476476 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:34.476479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:34.476564 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:34.476654 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:34.477385 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.478464 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.479784 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.480261 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.480352 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.483872 (+ 3520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.129:52457: BlockingRecv error: recv got EOF from 127.10.181.129:52457 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 13:10:34 dist-test-slave-42z9 krb5kdc[24066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.181.130@KRBTEST.COM for oryx/127.10.181.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 13:10:34.927536 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:34.919715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34049 (local address 127.10.181.130:33107)
0809 13:10:34.920165 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:34.920182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:34.920295 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:34.920569 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:34.920582 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:34.920963 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:34.921136 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.921145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.922133 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.922137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.923175 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:34.923190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.923975 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.923979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:34.924102 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.924635 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:34.924652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:34.927286 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.181.190:34049: Server oryx/127.10.181.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":237,"threads_started":1}
W20260809 13:10:34.927721 25322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:34.919810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:33107 (local address 127.10.181.190:34049)
0809 13:10:34.920020 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:34.920026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:34.920257 (+ 231us) server_negotiation.cc:407] Connection header received
0809 13:10:34.920385 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:34.920388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:34.920452 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:34.920529 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:34.921246 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.921984 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.923305 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:34.923848 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:34.923941 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:34.927490 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.181.130:33107: BlockingRecv error: recv got EOF from 127.10.181.130:33107 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:10:35.236536 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25081
I20260809 13:10:35.248780 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:35.237671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.10.181.129:46085)
0809 13:10:35.238326 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:35.238334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:35.238363 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:10:35.238424 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:35.238431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:35.238677 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:35.238830 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:35.240261 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.241333 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.242512 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.243108 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.243214 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:35.245742 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:35.245750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:35.245765 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:35.245804 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:35.248246 (+ 2442us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 13:10:35.248519 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37646: Request ticket server kudu/127.10.181.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":324,"threads_started":1}
W20260809 13:10:35.248895 25827 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37646: Request ticket server kudu/127.10.181.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 13:10:35.248899 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25081: Not authorized: Client connection negotiation failed: client connection to 127.10.181.129:46085: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 13:10:35.249191 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 25081
I20260809 13:10:35.278879 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25326
I20260809 13:10:35.290997 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:35.280032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.10.181.130:35485)
0809 13:10:35.280651 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:35.280659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:35.280692 (+ 33us) server_negotiation.cc:407] Connection header received
0809 13:10:35.280766 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:35.280773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:35.281060 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:35.281258 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:35.282604 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.283658 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.284892 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.285472 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.285601 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:35.288009 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:35.288018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:35.288033 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:35.288066 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:35.290349 (+ 2283us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 13:10:35.290673 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37928: Request ticket server kudu/127.10.181.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":294,"threads_started":1}
W20260809 13:10:35.291118 25829 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37928: Request ticket server kudu/127.10.181.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 13:10:35.291080 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25326: Not authorized: Client connection negotiation failed: client connection to 127.10.181.130:35485: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 13:10:35.291401 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 25326
I20260809 13:10:35.317886 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25548
I20260809 13:10:35.329793 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:35.318893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.10.181.131:38913)
0809 13:10:35.319501 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:35.319509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:35.319538 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:10:35.319608 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:35.319615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:35.319899 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:35.320066 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:35.321281 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.322361 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.323565 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:35.324161 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:35.324281 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:35.326820 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:35.326829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:35.326845 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:35.326880 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:35.329094 (+ 2214us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 13:10:35.329424 (+ 330us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43234: Request ticket server kudu/127.10.181.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":302,"threads_started":1}
W20260809 13:10:35.329823 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25548: Not authorized: Client connection negotiation failed: client connection to 127.10.181.131:38913: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.181.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 13:10:35.329906 25830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43234: Request ticket server kudu/127.10.181.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 13:10:35.329984 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 25548
I20260809 13:10:35.357414 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 24970
I20260809 13:10:35.358788 25070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:35.479637 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 24970
2026-08-09T13:10:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21292 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T13:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:35Z Disabled control of system clock
I20260809 13:10:35.574322 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37627
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:36439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:37627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 13:10:35.900302 25837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:35.900612 25837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:35.900735 25837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:35.907989 25837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:35.908093 25837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:35.908156 25837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:35.908231 25837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:35.908296 25837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:35.917727 25837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:37627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:37627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:35.919045 25837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:35.920907 25837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:35.932183 25845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:35.932150 25847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:35.932395 25854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:35.932838 25837 server_base.cc:1034] running on GCE node
I20260809 13:10:35.933668 25837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:35.934999 25837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:35.936194 25837 hybrid_clock.cc:648] HybridClock initialized: now 1786281035936138 us; error 45 us; skew 500 ppm
I20260809 13:10:35.936530 25837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:35.938722 25837 webserver.cc:460] Webserver started at http://127.10.181.190:36843/ using document root <none> and password file <none>
I20260809 13:10:35.939404 25837 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:35.939503 25837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:35.939827 25837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:35.942736 25837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "7c1894740db949f4b3ada81cfe1454df"
format_stamp: "Formatted at 2026-08-09 13:10:35 on dist-test-slave-42z9"
server_key: "d162760be9b3ef473ce2b4fc8044e442"
server_key_iv: "8c88cb568ac2d7d0124f402e501974d0"
server_key_version: "encryptionkey@0"
I20260809 13:10:35.943543 25837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "7c1894740db949f4b3ada81cfe1454df"
format_stamp: "Formatted at 2026-08-09 13:10:35 on dist-test-slave-42z9"
server_key: "d162760be9b3ef473ce2b4fc8044e442"
server_key_iv: "8c88cb568ac2d7d0124f402e501974d0"
server_key_version: "encryptionkey@0"
I20260809 13:10:35.948446 25837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:10:35.951683 25863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:35.952901 25837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 13:10:35.953071 25837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "7c1894740db949f4b3ada81cfe1454df"
format_stamp: "Formatted at 2026-08-09 13:10:35 on dist-test-slave-42z9"
server_key: "d162760be9b3ef473ce2b4fc8044e442"
server_key_iv: "8c88cb568ac2d7d0124f402e501974d0"
server_key_version: "encryptionkey@0"
I20260809 13:10:35.953266 25837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:35.973385 25837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:35.974241 25837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:35.974505 25837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:35.990983 25837 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:37627
I20260809 13:10:35.991007 25944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:37627 every 8 connection(s)
I20260809 13:10:35.992292 25837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:35.995829 25947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:35.996369 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 25837
I20260809 13:10:35.996524 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:10:35.997179 10966 external_mini_cluster.cc:1428] Setting key fb485c21c399c56d16c89ed6aa6ece68
I20260809 13:10:36.008793 25947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df: Bootstrap starting.
I20260809 13:10:36.010473 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:35.999144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39358 (local address 127.10.181.190:37627)
0809 13:10:36.000108 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:36.000122 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:36.000169 (+ 47us) server_negotiation.cc:407] Connection header received
0809 13:10:36.001799 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:36.001821 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:36.002259 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:36.002568 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:36.003195 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.004408 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.006248 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.006914 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.007043 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:36.008309 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:36.008316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:36.008329 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:36.008825 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:36.008899 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:36.008924 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:36.009038 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:36.009262 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 13:10:36.009583 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":353,"threads_started":1}
I20260809 13:10:36.012048 25947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:36.013348 25947 log.cc:824] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:36.015872 25947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df: No bootstrap required, opened a new log
I20260809 13:10:36.023603 25947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } }
I20260809 13:10:36.023913 25947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:36.023988 25947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1894740db949f4b3ada81cfe1454df, State: Initialized, Role: FOLLOWER
I20260809 13:10:36.024453 25947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } }
I20260809 13:10:36.024644 25947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:36.024745 25947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:36.024871 25947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:36.027262 25947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } }
I20260809 13:10:36.027700 25947 leader_election.cc:302] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c1894740db949f4b3ada81cfe1454df; no voters:
I20260809 13:10:36.028173 25947 leader_election.cc:288] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:10:36.028317 25955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:36.028944 25955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [term 1 LEADER]: Becoming Leader. State: Replica: 7c1894740db949f4b3ada81cfe1454df, State: Running, Role: LEADER
I20260809 13:10:36.029561 25955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } }
I20260809 13:10:36.030138 25947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:36.031986 25957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c1894740db949f4b3ada81cfe1454df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } } }
I20260809 13:10:36.032414 25957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:36.032219 25958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c1894740db949f4b3ada81cfe1454df. Latest consensus state: current_term: 1 leader_uuid: "7c1894740db949f4b3ada81cfe1454df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1894740db949f4b3ada81cfe1454df" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 37627 } } }
I20260809 13:10:36.032554 25958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:36.034611 25970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:36.038874 25970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:36.047097 25970 catalog_manager.cc:1347] Generated new cluster ID: dbd26860452340ffa1363d8441e1a967
I20260809 13:10:36.047204 25970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:36.121877 25970 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:10:36.123816 25970 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:36.137408 25970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c1894740db949f4b3ada81cfe1454df: Generated new TSK 0
I20260809 13:10:36.138415 25970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:10:36.157109 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--builtin_ntp_servers=127.10.181.148:36439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 13:10:36.490288 25988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:36.490556 25988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:36.490688 25988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:36.497763 25988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:36.497848 25988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:36.497982 25988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:36.506801 25988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:36.508038 25988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:36.509923 25988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:36.522810 25996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:36.522740 25997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:36.522740 26001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:36.523910 25988 server_base.cc:1034] running on GCE node
I20260809 13:10:36.524585 25988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:36.525553 25988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:36.526757 25988 hybrid_clock.cc:648] HybridClock initialized: now 1786281036526697 us; error 52 us; skew 500 ppm
I20260809 13:10:36.527071 25988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:36.532202 25988 webserver.cc:460] Webserver started at http://127.10.181.129:41121/ using document root <none> and password file <none>
I20260809 13:10:36.532915 25988 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:36.533026 25988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:36.533360 25988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:36.536396 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "92663b0295f647aa8d6e52af14ee372a"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "4b93f3cd05a810688d7420de1ea1fd86"
server_key_iv: "a05b0ef0f7f45b2d4e0e0d12c45470db"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.537220 25988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "92663b0295f647aa8d6e52af14ee372a"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "4b93f3cd05a810688d7420de1ea1fd86"
server_key_iv: "a05b0ef0f7f45b2d4e0e0d12c45470db"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.542382 25988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 13:10:36.545753 26009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:36.547027 25988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:36.547189 25988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "92663b0295f647aa8d6e52af14ee372a"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "4b93f3cd05a810688d7420de1ea1fd86"
server_key_iv: "a05b0ef0f7f45b2d4e0e0d12c45470db"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.547389 25988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:36.563607 25988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:36.564414 25988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:36.564698 25988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:36.565654 25988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:36.567116 25988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:36.567214 25988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:36.567325 25988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:36.567397 25988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:36.597404 25988 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:34127
I20260809 13:10:36.597492 26188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:34127 every 8 connection(s)
I20260809 13:10:36.598760 25988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:36.605273 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 25988
I20260809 13:10:36.605502 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:10:36.606318 10966 external_mini_cluster.cc:1428] Setting key 61b9d9e72f823a42a75e0af4348bd7ac
I20260809 13:10:36.609073 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--builtin_ntp_servers=127.10.181.148:36439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 13:10:36.611560 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:36.601348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:46723 (local address 127.10.181.190:37627)
0809 13:10:36.601811 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:36.601816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:36.602934 (+ 1118us) server_negotiation.cc:407] Connection header received
0809 13:10:36.604619 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:36.604624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:36.604693 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:36.604789 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:36.606220 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.606976 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.608819 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.609251 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.609374 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:36.610711 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:36.610716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:36.610719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:36.610737 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:36.610778 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:36.610790 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:36.610863 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:36.611101 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 13:10:36.611199 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":240,"threads_started":1}
I20260809 13:10:36.612553 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:36.601934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:37627 (local address 127.10.181.129:46723)
0809 13:10:36.602787 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:36.602828 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:36.604395 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:36.605053 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:36.605065 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:36.605200 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:36.606018 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:36.606045 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.607140 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.607144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:36.608667 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:36.608681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:36.609499 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:36.609508 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:36.609731 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:36.610387 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:36.610390 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:36.610432 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:36.610517 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:36.610879 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:36.610891 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:36.610898 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 13:10:36.611056 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 13:10:36.611461 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":259,"threads_started":1}
I20260809 13:10:36.614153 26189 heartbeater.cc:347] Connected to a master server at 127.10.181.190:37627
I20260809 13:10:36.614471 26189 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:36.615281 26189 heartbeater.cc:511] Master 127.10.181.190:37627 requested a full tablet report, sending...
I20260809 13:10:36.617651 25890 ts_manager.cc:194] Registered new tserver with Master: 92663b0295f647aa8d6e52af14ee372a (127.10.181.129:34127)
I20260809 13:10:36.619650 25890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.129:46723
W20260809 13:10:36.944324 26199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:36.944574 26199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:36.944661 26199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:36.951648 26199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:36.951733 26199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:36.951838 26199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:36.960537 26199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:36.961664 26199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:36.963359 26199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:36.975718 26205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:36.975696 26209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:36.975801 26211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:36.976802 26199 server_base.cc:1034] running on GCE node
I20260809 13:10:36.977377 26199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:36.978331 26199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:36.979532 26199 hybrid_clock.cc:648] HybridClock initialized: now 1786281036979478 us; error 48 us; skew 500 ppm
I20260809 13:10:36.979831 26199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:36.981950 26199 webserver.cc:460] Webserver started at http://127.10.181.130:37517/ using document root <none> and password file <none>
I20260809 13:10:36.982609 26199 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:36.982709 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:36.983022 26199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:36.985836 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "1c827dc72dfd401eaff053f143f7a139"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "df9f742e18ee5461a59bffa32782944c"
server_key_iv: "8669a086772f64de82a5be7a2234da86"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.986589 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "1c827dc72dfd401eaff053f143f7a139"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "df9f742e18ee5461a59bffa32782944c"
server_key_iv: "8669a086772f64de82a5be7a2234da86"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.991611 26199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:10:36.994969 26226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:36.996292 26199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:10:36.996441 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "1c827dc72dfd401eaff053f143f7a139"
format_stamp: "Formatted at 2026-08-09 13:10:36 on dist-test-slave-42z9"
server_key: "df9f742e18ee5461a59bffa32782944c"
server_key_iv: "8669a086772f64de82a5be7a2234da86"
server_key_version: "encryptionkey@0"
I20260809 13:10:36.996629 26199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:37.010215 26199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:37.010957 26199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:37.011229 26199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:37.012126 26199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:37.013531 26199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:37.013629 26199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:37.013736 26199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:37.013803 26199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:37.044699 26199 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:36753
I20260809 13:10:37.044749 26407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:36753 every 8 connection(s)
I20260809 13:10:37.046030 26199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:37.048221 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 26199
I20260809 13:10:37.048372 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:10:37.048883 10966 external_mini_cluster.cc:1428] Setting key f5b55e0432c47e4b8fb1d5890da8be66
I20260809 13:10:37.051994 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--builtin_ntp_servers=127.10.181.148:36439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 13:10:37.058519 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.048591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:42615 (local address 127.10.181.190:37627)
0809 13:10:37.048810 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.048816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.050333 (+ 1517us) server_negotiation.cc:407] Connection header received
0809 13:10:37.052346 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.052351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.052425 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.052523 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.053628 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.054387 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.055895 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.056276 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.056385 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.057755 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:37.057761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:37.057765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:37.057785 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:37.057826 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:37.057840 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:37.057928 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:37.058146 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 13:10:37.058238 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:10:37.059372 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.049115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:37627 (local address 127.10.181.130:42615)
0809 13:10:37.050170 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:37.050226 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:37.052064 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:37.052755 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:37.052763 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:37.052844 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:37.053409 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:37.053430 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.054545 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.054549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:37.055753 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:37.055766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.056509 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.056517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:37.056771 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.057422 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:37.057425 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:37.057497 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:37.057573 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:37.057946 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:37.057954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:37.057959 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:10:37.058094 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 13:10:37.058369 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":321,"threads_started":1}
I20260809 13:10:37.060632 26408 heartbeater.cc:347] Connected to a master server at 127.10.181.190:37627
I20260809 13:10:37.060909 26408 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:37.061691 26408 heartbeater.cc:511] Master 127.10.181.190:37627 requested a full tablet report, sending...
I20260809 13:10:37.062988 25890 ts_manager.cc:194] Registered new tserver with Master: 1c827dc72dfd401eaff053f143f7a139 (127.10.181.130:36753)
I20260809 13:10:37.063750 25890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.130:42615
W20260809 13:10:37.384869 26411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:37.385135 26411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:37.385270 26411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:37.392248 26411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:37.392347 26411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:37.392493 26411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:37.401259 26411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:37627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:37.402479 26411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:37.404203 26411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:37.416342 26422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:37.416369 26418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:37.416391 26419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:37.417338 26411 server_base.cc:1034] running on GCE node
I20260809 13:10:37.417930 26411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:37.418776 26411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:37.419967 26411 hybrid_clock.cc:648] HybridClock initialized: now 1786281037419914 us; error 44 us; skew 500 ppm
I20260809 13:10:37.420264 26411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:37.422382 26411 webserver.cc:460] Webserver started at http://127.10.181.131:41265/ using document root <none> and password file <none>
I20260809 13:10:37.423043 26411 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:37.423141 26411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:37.423449 26411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:37.426381 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "3916675a54304ecaa198ec1f189a6e65"
format_stamp: "Formatted at 2026-08-09 13:10:37 on dist-test-slave-42z9"
server_key: "e2062ad9bd910dc384dd29a454bd3bf1"
server_key_iv: "7600081d7d7586cd40d28001ddb5d251"
server_key_version: "encryptionkey@0"
I20260809 13:10:37.427138 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "3916675a54304ecaa198ec1f189a6e65"
format_stamp: "Formatted at 2026-08-09 13:10:37 on dist-test-slave-42z9"
server_key: "e2062ad9bd910dc384dd29a454bd3bf1"
server_key_iv: "7600081d7d7586cd40d28001ddb5d251"
server_key_version: "encryptionkey@0"
I20260809 13:10:37.431948 26411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:10:37.435237 26437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:37.436400 26411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 13:10:37.436546 26411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "3916675a54304ecaa198ec1f189a6e65"
format_stamp: "Formatted at 2026-08-09 13:10:37 on dist-test-slave-42z9"
server_key: "e2062ad9bd910dc384dd29a454bd3bf1"
server_key_iv: "7600081d7d7586cd40d28001ddb5d251"
server_key_version: "encryptionkey@0"
I20260809 13:10:37.436751 26411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:37.468760 26411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:37.469579 26411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:37.469851 26411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:37.470752 26411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:37.472067 26411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:37.472164 26411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:37.472272 26411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:37.472350 26411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:37.501062 26411 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:33737
I20260809 13:10:37.501129 26617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:33737 every 8 connection(s)
I20260809 13:10:37.502262 26411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:37.509052 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 26411
I20260809 13:10:37.509204 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:10:37.509775 10966 external_mini_cluster.cc:1428] Setting key c82c00f397bb27e9aef7038e7e9711db
I20260809 13:10:37.514286 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.504613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:52781 (local address 127.10.181.190:37627)
0809 13:10:37.504780 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.504784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.506051 (+ 1267us) server_negotiation.cc:407] Connection header received
0809 13:10:37.507687 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.507692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.507763 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.507846 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.508813 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.509928 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.511648 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.512272 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.512400 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.513644 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:37.513649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:37.513653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:37.513670 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:37.513700 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:37.513711 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:37.513775 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:37.513955 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 13:10:37.514032 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:10:37.515137 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.505057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:37627 (local address 127.10.181.131:52781)
0809 13:10:37.505912 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:37.505948 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:37.507468 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:37.508020 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:37.508027 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:37.508119 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:37.508655 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:37.508672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.510081 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.510085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:37.511506 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:37.511522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.512499 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.512505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:37.512773 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.513338 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:37.513340 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:37.513376 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:37.513467 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:37.513780 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:37.513786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:37.513791 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:10:37.513916 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 13:10:37.514191 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":250,"threads_started":1}
I20260809 13:10:37.516345 26619 heartbeater.cc:347] Connected to a master server at 127.10.181.190:37627
I20260809 13:10:37.516611 26619 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:37.517302 26619 heartbeater.cc:511] Master 127.10.181.190:37627 requested a full tablet report, sending...
I20260809 13:10:37.518633 25890 ts_manager.cc:194] Registered new tserver with Master: 3916675a54304ecaa198ec1f189a6e65 (127.10.181.131:33737)
I20260809 13:10:37.519397 25890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.131:52781
I20260809 13:10:37.524806 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 13:10:37.530699 26192 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:37.529036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.10.181.190:37627)
0809 13:10:37.529176 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.529180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.529716 (+ 536us) server_negotiation.cc:407] Connection header received
0809 13:10:37.529871 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.529875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.529945 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.530022 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.530458 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39368: BlockingRecv error: recv got EOF from 127.0.0.1:39368 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 13:10:37.531189 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25988
I20260809 13:10:37.538228 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.532006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.10.181.129:34127)
0809 13:10:37.532506 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.532512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.532541 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:10:37.532620 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.532626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.532809 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.532929 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.533237 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.534276 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.535465 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.535997 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.536099 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.536966 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:37.536973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:37.536985 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:37.537558 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:37.537612 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:37.537632 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:37.537710 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:37.537818 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 13:10:37.537938 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":257,"threads_started":1}
I20260809 13:10:37.539310 26183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:37.619447 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 25988
I20260809 13:10:37.644687 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 26199
I20260809 13:10:37.652660 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.645589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.10.181.130:36753)
0809 13:10:37.646149 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.646157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.646184 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:10:37.646256 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.646262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.646475 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.646636 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.647067 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.648092 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.649345 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.650074 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.650197 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.651212 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:37.651221 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:37.651235 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:37.651825 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:37.651874 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:37.651908 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:37.652013 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:37.652171 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 13:10:37.652324 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":275,"threads_started":1}
I20260809 13:10:37.653885 26399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:37.734264 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 26199
I20260809 13:10:37.757279 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 26411
I20260809 13:10:37.765745 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:37.758370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.10.181.131:33737)
0809 13:10:37.759010 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:37.759017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:37.759048 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:10:37.759113 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:37.759129 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:37.759352 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:37.759525 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:37.760007 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.761070 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.762470 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:37.763042 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:37.763174 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:37.764257 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:37.764266 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:37.764280 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:37.764861 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:37.764925 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:37.764946 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:37.765052 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:37.765208 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 13:10:37.765352 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":312,"threads_started":1}
I20260809 13:10:37.766968 26613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:37.847604 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 26411
I20260809 13:10:37.871075 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 25837
I20260809 13:10:37.872367 25938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:37.974920 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 25837
2026-08-09T13:10:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2490 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T13:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:38Z Disabled control of system clock
I20260809 13:10:38.064097 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46475
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 13:10:38.395062 26652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:38.395321 26652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:38.395453 26652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:38.402376 26652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:38.402480 26652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:38.402546 26652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:38.402623 26652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:38.402680 26652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:38.411391 26652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:38.412614 26652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:38.414373 26652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:38.425366 26659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:38.425413 26660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:38.425465 26662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:38.426514 26652 server_base.cc:1034] running on GCE node
I20260809 13:10:38.427224 26652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:38.428478 26652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:38.429670 26652 hybrid_clock.cc:648] HybridClock initialized: now 1786281038429626 us; error 32 us; skew 500 ppm
I20260809 13:10:38.429976 26652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:38.432062 26652 webserver.cc:460] Webserver started at http://127.10.181.190:39011/ using document root <none> and password file <none>
I20260809 13:10:38.432721 26652 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:38.432817 26652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:38.433128 26652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:38.436017 26652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "b4e23be2cbdd4fb295a25a77156b9578"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "fdb53bc4bd1782ed33100e92d2ce50bd"
server_key_iv: "57ebbf7a814117e78c80ed22febea36c"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.436834 26652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "b4e23be2cbdd4fb295a25a77156b9578"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "fdb53bc4bd1782ed33100e92d2ce50bd"
server_key_iv: "57ebbf7a814117e78c80ed22febea36c"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.442005 26652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:10:38.445262 26672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:38.446501 26652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:38.446669 26652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "b4e23be2cbdd4fb295a25a77156b9578"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "fdb53bc4bd1782ed33100e92d2ce50bd"
server_key_iv: "57ebbf7a814117e78c80ed22febea36c"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.446864 26652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:38.452066 26652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:38.452778 26652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:38.453048 26652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:38.469184 26652 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46475
I20260809 13:10:38.469221 26754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46475 every 8 connection(s)
I20260809 13:10:38.470420 26652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:38.474030 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:38.477190 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 26652
I20260809 13:10:38.477411 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:10:38.478232 10966 external_mini_cluster.cc:1428] Setting key d79f11ee973da8c7193a24b8f8e47a97
I20260809 13:10:38.486124 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:38.480230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.10.181.190:46475)
0809 13:10:38.481139 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:38.481150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:38.481190 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:10:38.482761 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:38.482782 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:38.483168 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:38.483481 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:38.484531 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:38.484537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:38.484549 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:38.484563 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:38.484634 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:38.484661 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:38.484811 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:38.484941 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 13:10:38.485171 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":329,"threads_started":1}
I20260809 13:10:38.486588 26755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578: Bootstrap starting.
I20260809 13:10:38.489980 26755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:38.491046 26755 log.cc:824] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:38.493412 26755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578: No bootstrap required, opened a new log
I20260809 13:10:38.500267 26755 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } }
I20260809 13:10:38.500590 26755 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:38.500696 26755 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e23be2cbdd4fb295a25a77156b9578, State: Initialized, Role: FOLLOWER
I20260809 13:10:38.501225 26755 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } }
I20260809 13:10:38.501431 26755 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:38.501582 26755 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:38.501718 26755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:38.503654 26755 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } }
I20260809 13:10:38.504066 26755 leader_election.cc:302] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4e23be2cbdd4fb295a25a77156b9578; no voters:
I20260809 13:10:38.504585 26755 leader_election.cc:288] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:10:38.504707 26760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:38.505366 26760 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [term 1 LEADER]: Becoming Leader. State: Replica: b4e23be2cbdd4fb295a25a77156b9578, State: Running, Role: LEADER
I20260809 13:10:38.505870 26760 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } }
I20260809 13:10:38.506461 26755 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:38.508599 26764 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4e23be2cbdd4fb295a25a77156b9578. Latest consensus state: current_term: 1 leader_uuid: "b4e23be2cbdd4fb295a25a77156b9578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } } }
I20260809 13:10:38.508543 26763 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4e23be2cbdd4fb295a25a77156b9578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e23be2cbdd4fb295a25a77156b9578" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46475 } } }
I20260809 13:10:38.508958 26764 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:38.510650 26763 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578 [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:38.511786 26775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:38.516656 26775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:38.525120 26775 catalog_manager.cc:1347] Generated new cluster ID: 55548fbe767b45bb9b02b4539dc80574
I20260809 13:10:38.525224 26775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:38.540182 26775 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:10:38.540555 26775 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:38.546877 26775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4e23be2cbdd4fb295a25a77156b9578: Generated new TSK 0
I20260809 13:10:38.547806 26775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:10:38.560032 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--builtin_ntp_servers=127.10.181.148:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 13:10:38.894253 26793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:38.894536 26793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:38.894667 26793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:38.901677 26793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:38.901777 26793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:38.901928 26793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:38.910724 26793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:38.911990 26793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:38.913851 26793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:38.926148 26801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:38.926174 26806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:38.926424 26803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:38.927096 26793 server_base.cc:1034] running on GCE node
I20260809 13:10:38.927619 26793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:38.928447 26793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:38.929622 26793 hybrid_clock.cc:648] HybridClock initialized: now 1786281038929577 us; error 44 us; skew 500 ppm
I20260809 13:10:38.929903 26793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:38.931950 26793 webserver.cc:460] Webserver started at http://127.10.181.129:45435/ using document root <none> and password file <none>
I20260809 13:10:38.932580 26793 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:38.932672 26793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:38.932969 26793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:38.935801 26793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "46cdb359ae524aab95317da83038d8e7"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "ab40b515d29c8fe6b70b96ce46c10f40"
server_key_iv: "100ae553b59e4a354bf29b05b021afab"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.936633 26793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "46cdb359ae524aab95317da83038d8e7"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "ab40b515d29c8fe6b70b96ce46c10f40"
server_key_iv: "100ae553b59e4a354bf29b05b021afab"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.941421 26793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:10:38.944569 26814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:38.945849 26793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260809 13:10:38.946007 26793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "46cdb359ae524aab95317da83038d8e7"
format_stamp: "Formatted at 2026-08-09 13:10:38 on dist-test-slave-42z9"
server_key: "ab40b515d29c8fe6b70b96ce46c10f40"
server_key_iv: "100ae553b59e4a354bf29b05b021afab"
server_key_version: "encryptionkey@0"
I20260809 13:10:38.946202 26793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:38.951578 26793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:38.952299 26793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:38.952538 26793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:38.953404 26793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:38.954727 26793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:38.954823 26793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:38.954926 26793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:38.954996 26793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:38.984524 26793 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:40279
I20260809 13:10:38.984580 27001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:40279 every 8 connection(s)
I20260809 13:10:38.985837 26793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:38.987409 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 26793
I20260809 13:10:38.987596 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:10:38.988301 10966 external_mini_cluster.cc:1428] Setting key 816a9f3ff8b6a5cc9d21bce46ceb256a
I20260809 13:10:38.991266 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--builtin_ntp_servers=127.10.181.148:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 13:10:38.994431 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:38.988312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:42685 (local address 127.10.181.190:46475)
0809 13:10:38.988795 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:38.988801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:38.991489 (+ 2688us) server_negotiation.cc:407] Connection header received
0809 13:10:38.992336 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:38.992341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:38.992403 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:38.992507 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:38.993751 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:38.993756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:38.993760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:38.993780 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:38.993834 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:38.993849 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:38.993965 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:38.994101 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 13:10:38.994161 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260809 13:10:38.995214 27007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:38.988855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46475 (local address 127.10.181.129:42685)
0809 13:10:38.990300 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:38.990348 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:38.991933 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:38.992688 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:38.992695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:38.992779 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:38.993424 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:38.993427 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:38.993492 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:38.993583 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:38.993934 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:38.993940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:38.993945 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:10:38.994150 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 13:10:38.994334 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":633,"threads_started":1}
I20260809 13:10:38.996623 27003 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46475
I20260809 13:10:38.996894 27003 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:38.997488 27003 heartbeater.cc:511] Master 127.10.181.190:46475 requested a full tablet report, sending...
I20260809 13:10:38.999434 26695 ts_manager.cc:194] Registered new tserver with Master: 46cdb359ae524aab95317da83038d8e7 (127.10.181.129:40279)
W20260809 13:10:39.328745 27008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:39.329016 27008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:39.329149 27008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:39.336218 27008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:39.336328 27008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:39.336475 27008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:39.345306 27008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:39.346588 27008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:39.348341 27008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:39.360872 27016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:39.360904 27015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:39.361128 27018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:39.361742 27008 server_base.cc:1034] running on GCE node
I20260809 13:10:39.362324 27008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:39.363209 27008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:39.364406 27008 hybrid_clock.cc:648] HybridClock initialized: now 1786281039364358 us; error 43 us; skew 500 ppm
I20260809 13:10:39.364724 27008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:39.366891 27008 webserver.cc:460] Webserver started at http://127.10.181.130:33743/ using document root <none> and password file <none>
I20260809 13:10:39.367586 27008 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:39.367688 27008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:39.368000 27008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:39.370863 27008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "5a601f9f43d9439f8159a2a48fdd56d2"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "97b38f5e9524e099f75afe8cf470ad25"
server_key_iv: "3632fa353bf2e40f188b0e2a6eae24a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.371621 27008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "5a601f9f43d9439f8159a2a48fdd56d2"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "97b38f5e9524e099f75afe8cf470ad25"
server_key_iv: "3632fa353bf2e40f188b0e2a6eae24a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.376420 27008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:10:39.379665 27028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.380950 27008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:10:39.381110 27008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "5a601f9f43d9439f8159a2a48fdd56d2"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "97b38f5e9524e099f75afe8cf470ad25"
server_key_iv: "3632fa353bf2e40f188b0e2a6eae24a2"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.381302 27008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:39.386557 27008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:39.387302 27008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:39.387543 27008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:39.388393 27008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:39.389750 27008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:39.389837 27008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.389915 27008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:39.389967 27008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.419694 27008 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43395
I20260809 13:10:39.419766 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43395 every 8 connection(s)
I20260809 13:10:39.421028 27008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:39.428848 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:39.423587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:45409 (local address 127.10.181.190:46475)
0809 13:10:39.423773 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:39.423777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:39.425148 (+ 1371us) server_negotiation.cc:407] Connection header received
0809 13:10:39.426864 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:39.426870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:39.426931 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:39.427024 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:39.428287 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:39.428291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:39.428294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:39.428309 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:39.428337 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:39.428348 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:39.428439 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:39.428576 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 13:10:39.428624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:10:39.429342 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 27008
I20260809 13:10:39.429531 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:10:39.429806 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:39.424095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46475 (local address 127.10.181.130:45409)
0809 13:10:39.424993 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:39.425028 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:39.426603 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:39.427295 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:39.427303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:39.427390 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:39.428016 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:39.428019 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:39.428057 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:39.428147 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:39.428422 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:39.428427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:39.428432 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:10:39.428646 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 13:10:39.428836 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":273,"threads_started":1}
I20260809 13:10:39.430006 10966 external_mini_cluster.cc:1428] Setting key bd99a574bf0ecab3dd70d4a6de5a870f
I20260809 13:10:39.431156 27214 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46475
I20260809 13:10:39.431425 27214 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:39.431980 27214 heartbeater.cc:511] Master 127.10.181.190:46475 requested a full tablet report, sending...
I20260809 13:10:39.432654 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--builtin_ntp_servers=127.10.181.148:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 13:10:39.433348 26695 ts_manager.cc:194] Registered new tserver with Master: 5a601f9f43d9439f8159a2a48fdd56d2 (127.10.181.130:43395)
W20260809 13:10:39.772835 27218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:39.773093 27218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:39.773236 27218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:39.780475 27218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:39.780573 27218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:39.780710 27218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:39.789714 27218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:39.790946 27218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:39.792658 27218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:39.805151 27231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:39.805086 27236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:39.805209 27232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:39.806268 27218 server_base.cc:1034] running on GCE node
I20260809 13:10:39.806784 27218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:39.807646 27218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:39.808841 27218 hybrid_clock.cc:648] HybridClock initialized: now 1786281039808791 us; error 33 us; skew 500 ppm
I20260809 13:10:39.809144 27218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:10:39.811237 27218 webserver.cc:460] Webserver started at http://127.10.181.131:36897/ using document root <none> and password file <none>
I20260809 13:10:39.811932 27218 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:39.812026 27218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:39.812326 27218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:39.815070 27218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "562e2077a65741979b5de21a3b38b66a"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "a3076cc32863b2838b20e459ed0854a1"
server_key_iv: "ddf485e5ebbce84c45e1944c2e703934"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.815937 27218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "562e2077a65741979b5de21a3b38b66a"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "a3076cc32863b2838b20e459ed0854a1"
server_key_iv: "ddf485e5ebbce84c45e1944c2e703934"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.822726 27218 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 13:10:39.826741 27242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.828070 27218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:10:39.828207 27218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "562e2077a65741979b5de21a3b38b66a"
format_stamp: "Formatted at 2026-08-09 13:10:39 on dist-test-slave-42z9"
server_key: "a3076cc32863b2838b20e459ed0854a1"
server_key_iv: "ddf485e5ebbce84c45e1944c2e703934"
server_key_version: "encryptionkey@0"
I20260809 13:10:39.828389 27218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:39.833678 27218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:39.834367 27218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:39.834589 27218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:39.835392 27218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:39.836700 27218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:39.836786 27218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.836863 27218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:39.836915 27218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:39.866168 27218 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:44527
I20260809 13:10:39.866240 27416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:44527 every 8 connection(s)
I20260809 13:10:39.867427 27218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:39.869953 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 27218
I20260809 13:10:39.870139 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:10:39.870543 10966 external_mini_cluster.cc:1428] Setting key 892d46e9024998a9a10ace73c7227e8b
I20260809 13:10:39.875464 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:39.869883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:37317 (local address 127.10.181.190:46475)
0809 13:10:39.870135 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:39.870142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:39.871603 (+ 1461us) server_negotiation.cc:407] Connection header received
0809 13:10:39.873628 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:39.873634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:39.873687 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:39.873766 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:39.874953 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:39.874957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:39.874960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:39.874972 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:39.874999 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:39.875009 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:39.875096 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:39.875221 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 13:10:39.875269 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:10:39.876304 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:39.870410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46475 (local address 127.10.181.131:37317)
0809 13:10:39.871418 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:39.871472 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:39.873320 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:39.873991 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:39.873998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:39.874084 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:39.874705 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:10:39.874708 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:39.874744 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:10:39.874827 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:39.875067 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:39.875072 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:39.875076 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 13:10:39.875282 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 13:10:39.875462 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":288,"threads_started":1}
I20260809 13:10:39.877398 27417 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46475
I20260809 13:10:39.877707 27417 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:39.878255 27417 heartbeater.cc:511] Master 127.10.181.190:46475 requested a full tablet report, sending...
I20260809 13:10:39.879559 26695 ts_manager.cc:194] Registered new tserver with Master: 562e2077a65741979b5de21a3b38b66a (127.10.181.131:44527)
I20260809 13:10:39.885743 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 13:10:39.891703 27006 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:10:39.890140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.10.181.190:46475)
0809 13:10:39.890295 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:39.890300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:39.890789 (+ 489us) server_negotiation.cc:407] Connection header received
0809 13:10:39.890947 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:39.890952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:39.891009 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:39.891087 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:39.891460 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48234: BlockingRecv error: recv got EOF from 127.0.0.1:48234 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:10:39.892436 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 26793
I20260809 13:10:39.895720 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:39.893408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.10.181.129:40279)
0809 13:10:39.893965 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:39.893971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:39.893995 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:10:39.894059 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:39.894064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:39.894231 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:39.894346 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:39.895147 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:39.895153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:39.895162 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:39.895181 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:39.895219 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:39.895231 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:39.895333 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:39.895405 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 13:10:39.895480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":264,"threads_started":1}
I20260809 13:10:39.896863 26994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:39.977305 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 26793
I20260809 13:10:40.001024 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 27008
I20260809 13:10:40.004655 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:40.001935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.10.181.130:43395)
0809 13:10:40.002499 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:40.002506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:40.002535 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:10:40.002599 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:40.002606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:40.002816 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:40.002958 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:40.003927 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:40.003934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:40.003945 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:40.003963 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:40.004003 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:40.004016 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:40.004137 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:40.004283 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 13:10:40.004400 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":250,"threads_started":1}
I20260809 13:10:40.005959 27206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:40.089293 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 27008
I20260809 13:10:40.112412 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 27218
I20260809 13:10:40.116765 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:40.113578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.10.181.131:44527)
0809 13:10:40.114403 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:40.114410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:40.114444 (+ 34us) server_negotiation.cc:407] Connection header received
0809 13:10:40.114537 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:40.114544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:40.114759 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:40.114890 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:40.115945 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:40.115952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:40.115964 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:10:40.115977 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:40.116020 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:10:40.116034 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:40.116175 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:40.116281 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 13:10:40.116369 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":460,"threads_started":1}
I20260809 13:10:40.118014 27409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:40.198437 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 27218
I20260809 13:10:40.220715 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 26652
I20260809 13:10:40.221930 26749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:40.326565 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 26652
2026-08-09T13:10:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2351 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:10:40 dist-test-slave-42z9 krb5kdc[27443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:10:40 dist-test-slave-42z9 krb5kdc[27443](info): set up 2 sockets
Aug 09 13:10:40 dist-test-slave-42z9 krb5kdc[27443](info): commencing operation
krb5kdc: starting...
W20260809 13:10:41.563274 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:10:41 dist-test-slave-42z9 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:10:41.815047 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33855
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:33855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:42.140575 27460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:42.140853 27460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:42.140983 27460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:42.147919 27460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:10:42.148015 27460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:42.148089 27460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:42.148162 27460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:10:42.148219 27460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:10:42.157655 27460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:33855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:42.158869 27460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:42.160293 27460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:42.172056 27468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:42.172196 27466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:42.172412 27465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:42.172978 27460 server_base.cc:1034] running on GCE node
I20260809 13:10:42.173732 27460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:42.174985 27460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:42.176193 27460 hybrid_clock.cc:648] HybridClock initialized: now 1786281042176139 us; error 41 us; skew 500 ppm
I20260809 13:10:42.176514 27460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:42 dist-test-slave-42z9 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281042, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:42.182036 27460 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:10:42.183360 27460 webserver.cc:460] Webserver started at http://127.10.181.190:35667/ using document root <none> and password file <none>
I20260809 13:10:42.183980 27460 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:42.184080 27460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:42.184324 27460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:42.187290 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "f4e53d396fc444c98fd1042761e7fb7d"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "13dcc226971dbeffa3dcd95df6a663fe"
server_key_iv: "f9598d6579e04416ed5cea0bd9985df9"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.188009 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "f4e53d396fc444c98fd1042761e7fb7d"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "13dcc226971dbeffa3dcd95df6a663fe"
server_key_iv: "f9598d6579e04416ed5cea0bd9985df9"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.192921 27460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:10:42.196226 27483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:42.197540 27460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:10:42.197678 27460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "f4e53d396fc444c98fd1042761e7fb7d"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "13dcc226971dbeffa3dcd95df6a663fe"
server_key_iv: "f9598d6579e04416ed5cea0bd9985df9"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.197832 27460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:42.220642 27460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:42.224671 27460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:42.224908 27460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:42.241842 27460 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:33855
I20260809 13:10:42.241889 27568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:33855 every 8 connection(s)
I20260809 13:10:42.243155 27460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:10:42.246764 27572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:42.247938 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 27460
I20260809 13:10:42.248158 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:10:42.248776 10966 external_mini_cluster.cc:1428] Setting key 39f6e80cbd3794d589f6f377dc8c49d4
I20260809 13:10:42.260473 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: Bootstrap starting.
Aug 09 13:10:42 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:42.263378 27572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:42.264376 27572 log.cc:824] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:42.267354 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: No bootstrap required, opened a new log
I20260809 13:10:42.271255 27577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:42.250724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.10.181.190:33855)
0809 13:10:42.251537 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:42.251549 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:42.251593 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:10:42.253155 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:42.253178 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:42.253590 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:42.253894 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:42.255811 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.257820 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.259451 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.260104 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.260275 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:42.264116 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:42.264125 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:42.264139 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:42.264174 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:42.267361 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:42.268025 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:42.268032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:42.268038 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:42.268213 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:42.268586 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:42.268590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:42.268592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:42.269111 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:42.269426 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:42.269883 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 13:10:42.270280 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":372,"threads_started":1}
I20260809 13:10:42.275442 27572 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } }
I20260809 13:10:42.275779 27572 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:42.275866 27572 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e53d396fc444c98fd1042761e7fb7d, State: Initialized, Role: FOLLOWER
I20260809 13:10:42.276350 27572 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } }
I20260809 13:10:42.276561 27572 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:10:42.276690 27572 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:10:42.276834 27572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:42.278934 27572 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } }
I20260809 13:10:42.279372 27572 leader_election.cc:302] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4e53d396fc444c98fd1042761e7fb7d; no voters:
I20260809 13:10:42.279974 27572 leader_election.cc:288] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:10:42.280099 27580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:42.280893 27580 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [term 1 LEADER]: Becoming Leader. State: Replica: f4e53d396fc444c98fd1042761e7fb7d, State: Running, Role: LEADER
I20260809 13:10:42.281500 27580 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } }
I20260809 13:10:42.282199 27572 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:10:42.284422 27581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4e53d396fc444c98fd1042761e7fb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } } }
I20260809 13:10:42.284814 27581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:42.285264 27582 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4e53d396fc444c98fd1042761e7fb7d. Latest consensus state: current_term: 1 leader_uuid: "f4e53d396fc444c98fd1042761e7fb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e53d396fc444c98fd1042761e7fb7d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33855 } } }
I20260809 13:10:42.285647 27582 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d [sys.catalog]: This master's current role is: LEADER
I20260809 13:10:42.286772 27593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:10:42.291901 27593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:10:42.299455 27593 catalog_manager.cc:1347] Generated new cluster ID: 55943346883442cd84b54a21ab4e406c
I20260809 13:10:42.299556 27593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:10:42.345487 27593 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:10:42.347354 27593 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:10:42.362990 27593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: Generated new TSK 0
I20260809 13:10:42.363813 27593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:10:42.426405 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:42.758016 27615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:42.758279 27615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:42.758380 27615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:42.765368 27615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:42.765501 27615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:42.765626 27615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:10:42.775749 27615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:42.776924 27615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:42.778208 27615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:42.791420 27625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:42.791290 27630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:42.791460 27632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:42.792258 27615 server_base.cc:1034] running on GCE node
I20260809 13:10:42.792811 27615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:42.793762 27615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:42.794965 27615 hybrid_clock.cc:648] HybridClock initialized: now 1786281042794917 us; error 43 us; skew 500 ppm
I20260809 13:10:42.795244 27615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:42 dist-test-slave-42z9 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281042, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:42.798321 27615 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:10:42.799659 27615 webserver.cc:460] Webserver started at http://127.10.181.129:33271/ using document root <none> and password file <none>
I20260809 13:10:42.800282 27615 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:42.800383 27615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:42.800629 27615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:42.803541 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "4c875b93829f4e0d986fff79ad5559ee"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "89d6047a745c92f8dcd37327393f1a7e"
server_key_iv: "008f4b1eb7a68b49ced1e3fa8d0a751b"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.804262 27615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "4c875b93829f4e0d986fff79ad5559ee"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "89d6047a745c92f8dcd37327393f1a7e"
server_key_iv: "008f4b1eb7a68b49ced1e3fa8d0a751b"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.809036 27615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 13:10:42.812275 27639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:42.813620 27615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:10:42.813774 27615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "4c875b93829f4e0d986fff79ad5559ee"
format_stamp: "Formatted at 2026-08-09 13:10:42 on dist-test-slave-42z9"
server_key: "89d6047a745c92f8dcd37327393f1a7e"
server_key_iv: "008f4b1eb7a68b49ced1e3fa8d0a751b"
server_key_version: "encryptionkey@0"
I20260809 13:10:42.813951 27615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:42.825520 27615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:42.829011 27615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:42.829290 27615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:42.830191 27615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:42.831459 27615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:42.831539 27615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:42.831660 27615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:42.831714 27615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:42.862179 27615 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:33985
I20260809 13:10:42.862257 27815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:33985 every 8 connection(s)
I20260809 13:10:42.863547 27615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:10:42.870440 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 27615
I20260809 13:10:42.870657 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:10:42.871277 10966 external_mini_cluster.cc:1428] Setting key a3fc2e505e76b8d2f6f9590d13153054
Aug 09 13:10:42 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281042, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:42.882781 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:42.866014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:58605 (local address 127.10.181.190:33855)
0809 13:10:42.866429 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:42.866434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:42.867409 (+ 975us) server_negotiation.cc:407] Connection header received
0809 13:10:42.869171 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:42.869176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:42.869252 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:42.869347 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:42.870883 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.871654 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.873364 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.873795 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.873909 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:42.877321 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:42.877327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:42.877330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:42.877361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:42.879760 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:42.880450 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:42.880454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:42.880456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:42.880582 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:42.880990 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:42.880995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:42.880998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:42.881249 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:42.881380 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:42.882174 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 13:10:42.882417 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":207,"threads_started":1}
I20260809 13:10:42.883687 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:42.866574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33855 (local address 127.10.181.129:58605)
0809 13:10:42.867267 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:42.867303 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:42.868908 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:42.869621 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:42.869630 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:42.870137 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:42.870713 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:42.870737 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.871798 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.871803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:42.873224 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:42.873237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:42.874012 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:42.874020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:42.874251 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:42.874856 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:42.874879 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:42.877117 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:42.879901 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:42.879911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:42.879927 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:42.880338 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:42.880684 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:42.880689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:42.880693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:42.880883 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:42.881372 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:42.881379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:42.881756 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 13:10:42.882016 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 13:10:42.882534 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":262,"threads_started":1}
I20260809 13:10:42.885321 27816 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33855
I20260809 13:10:42.885651 27816 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:42.886421 27816 heartbeater.cc:511] Master 127.10.181.190:33855 requested a full tablet report, sending...
I20260809 13:10:42.888546 27509 ts_manager.cc:194] Registered new tserver with Master: 4c875b93829f4e0d986fff79ad5559ee (127.10.181.129:33985)
I20260809 13:10:42.890031 27509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:58605
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:10:42.932313 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:43.267491 27826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:43.267742 27826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:43.267840 27826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:43.274823 27826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:43.274923 27826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:43.275048 27826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:10:43.284468 27826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:43.285663 27826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:43.287043 27826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:43.299499 27840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:43.299559 27839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:43.299533 27842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:43.300345 27826 server_base.cc:1034] running on GCE node
I20260809 13:10:43.300946 27826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:43.301854 27826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:43.303061 27826 hybrid_clock.cc:648] HybridClock initialized: now 1786281043302997 us; error 56 us; skew 500 ppm
I20260809 13:10:43.303349 27826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:43 dist-test-slave-42z9 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:43.306267 27826 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:10:43.307578 27826 webserver.cc:460] Webserver started at http://127.10.181.130:38655/ using document root <none> and password file <none>
I20260809 13:10:43.308177 27826 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:43.308276 27826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:43.308516 27826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:43.311285 27826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "8aa5db7743e245068e93d459fcc93039"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "2af975fbefe6a497df9a6c6614295945"
server_key_iv: "4c0cb1c12956cfc1249025376082b03c"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.312028 27826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "8aa5db7743e245068e93d459fcc93039"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "2af975fbefe6a497df9a6c6614295945"
server_key_iv: "4c0cb1c12956cfc1249025376082b03c"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.316931 27826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 13:10:43.320387 27851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.321681 27826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:10:43.321808 27826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "8aa5db7743e245068e93d459fcc93039"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "2af975fbefe6a497df9a6c6614295945"
server_key_iv: "4c0cb1c12956cfc1249025376082b03c"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.321959 27826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:43.332566 27826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:43.336410 27826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:43.336647 27826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:43.337525 27826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:43.338745 27826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:43.338816 27826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.338901 27826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:43.338943 27826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.367293 27826 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:37933
I20260809 13:10:43.367336 28032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:37933 every 8 connection(s)
I20260809 13:10:43.368521 27826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:10:43.378145 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 27826
I20260809 13:10:43.378320 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:10:43.378734 10966 external_mini_cluster.cc:1428] Setting key 00d35fd1c5cc8ebdf5b0464c3e03736f
Aug 09 13:10:43 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:43.387703 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.370989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:35301 (local address 127.10.181.190:33855)
0809 13:10:43.371146 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.371150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.372270 (+ 1120us) server_negotiation.cc:407] Connection header received
0809 13:10:43.374086 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.374092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.374184 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.374276 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:43.375705 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.376439 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.378033 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.378545 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.378654 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.381992 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:43.381997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:43.382000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:43.382033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:43.384652 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.385343 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.385347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.385349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.385519 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.385932 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.385938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.385941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.386274 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:43.386442 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.387204 (+ 762us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.387416 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 13:10:43.388831 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.371403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33855 (local address 127.10.181.130:35301)
0809 13:10:43.372110 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:43.372154 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:43.373816 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:43.374505 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:43.374513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:43.374997 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:43.375533 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.375550 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.376588 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.376592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.377886 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.377905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.378774 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.378783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.379017 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.379640 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:43.379662 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:43.381781 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:43.384782 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:43.384792 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:43.384806 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:43.385219 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:43.385624 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:43.385629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:43.385633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:43.385815 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:43.386433 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:43.386441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:43.386816 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 13:10:43.387100 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 13:10:43.387620 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":273,"threads_started":1}
I20260809 13:10:43.390232 28037 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33855
I20260809 13:10:43.390522 28037 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:43.391201 28037 heartbeater.cc:511] Master 127.10.181.190:33855 requested a full tablet report, sending...
I20260809 13:10:43.392578 27509 ts_manager.cc:194] Registered new tserver with Master: 8aa5db7743e245068e93d459fcc93039 (127.10.181.130:37933)
I20260809 13:10:43.393347 27509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:35301
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:10:43.437371 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:10:43.766525 28046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:10:43.766803 28046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:10:43.766945 28046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:10:43.774219 28046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:10:43.774339 28046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:10:43.774501 28046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:10:43.784385 28046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:33855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:10:43.785759 28046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:10:43.787153 28046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:10:43.800055 28053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:43.800094 28056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:10:43.800122 28054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:10:43.800897 28046 server_base.cc:1034] running on GCE node
I20260809 13:10:43.801571 28046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:10:43.802551 28046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:10:43.803743 28046 hybrid_clock.cc:648] HybridClock initialized: now 1786281043803682 us; error 50 us; skew 500 ppm
I20260809 13:10:43.804049 28046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:10:43 dist-test-slave-42z9 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:10:43.807221 28046 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:10:43.808670 28046 webserver.cc:460] Webserver started at http://127.10.181.131:44641/ using document root <none> and password file <none>
I20260809 13:10:43.809355 28046 fs_manager.cc:359] Metadata directory not provided
I20260809 13:10:43.809500 28046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:10:43.809804 28046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:10:43.812791 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "50bde286bc854e30b469ce059ebb5ebb"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "87c5053c36c442bc1c3b56ce9ea50306"
server_key_iv: "bbe67e5547e86e0b94d8eb9828f6dfea"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.813643 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "50bde286bc854e30b469ce059ebb5ebb"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "87c5053c36c442bc1c3b56ce9ea50306"
server_key_iv: "bbe67e5547e86e0b94d8eb9828f6dfea"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.818884 28046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 13:10:43.822355 28074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.823697 28046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:10:43.823846 28046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "50bde286bc854e30b469ce059ebb5ebb"
format_stamp: "Formatted at 2026-08-09 13:10:43 on dist-test-slave-42z9"
server_key: "87c5053c36c442bc1c3b56ce9ea50306"
server_key_iv: "bbe67e5547e86e0b94d8eb9828f6dfea"
server_key_version: "encryptionkey@0"
I20260809 13:10:43.824023 28046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:10:43.845093 28046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:10:43.849491 28046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:10:43.849779 28046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:10:43.850780 28046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:10:43.852161 28046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:10:43.852247 28046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.852368 28046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:10:43.852422 28046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:10:43.883560 28046 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41557
I20260809 13:10:43.883605 28251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41557 every 8 connection(s)
I20260809 13:10:43.884855 28046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:10:43.890259 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 28046
I20260809 13:10:43.890444 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:10:43.890868 10966 external_mini_cluster.cc:1428] Setting key adef2f161cee689636117ce4b48f292c
I20260809 13:10:43.892877 27816 heartbeater.cc:503] Master 127.10.181.190:33855 was elected leader, sending a full tablet report...
Aug 09 13:10:43 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281043, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:10:43.904673 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.887663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:47167 (local address 127.10.181.190:33855)
0809 13:10:43.887860 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.887865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.889056 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 13:10:43.891154 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.891161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.891256 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.891368 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:43.893128 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.894257 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.895920 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.896335 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.896427 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.899653 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:43.899658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:43.899661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:43.899692 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:43.902150 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.902850 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.902853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.902855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.902971 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.903307 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.903309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.903311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.903552 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:43.903653 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.904216 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.904366 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:10:43.905889 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.888145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:33855 (local address 127.10.181.131:47167)
0809 13:10:43.888879 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:43.888927 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:43.890827 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:43.891624 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:43.891635 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:43.892195 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:10:43.892904 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.892927 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.894417 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.894422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.895790 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.895804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.896527 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.896532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.896720 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.897288 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:10:43.897316 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:10:43.899463 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:10:43.902289 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:43.902296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:43.902307 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:43.902765 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:43.903047 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:10:43.903050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:10:43.903052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:10:43.903231 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:10:43.903641 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:10:43.903646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:10:43.903923 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 13:10:43.904141 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 13:10:43.904607 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":269,"threads_started":1}
I20260809 13:10:43.907100 28256 heartbeater.cc:347] Connected to a master server at 127.10.181.190:33855
I20260809 13:10:43.907353 28256 heartbeater.cc:464] Registering TS with master...
I20260809 13:10:43.908025 28256 heartbeater.cc:511] Master 127.10.181.190:33855 requested a full tablet report, sending...
I20260809 13:10:43.909315 27508 ts_manager.cc:194] Registered new tserver with Master: 50bde286bc854e30b469ce059ebb5ebb (127.10.181.131:41557)
I20260809 13:10:43.910164 27508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:47167
I20260809 13:10:43.918098 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:10:43.933832 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.922389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.10.181.190:33855)
0809 13:10:43.922554 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.922560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.922960 (+ 400us) server_negotiation.cc:407] Connection header received
0809 13:10:43.923137 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.923142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.923207 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.923284 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:43.924671 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.925482 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.926703 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.927126 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.927245 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.929460 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:43.929468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:43.929472 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:43.929510 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:43.931744 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.932292 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.932295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.932297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.932416 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:43.932713 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:43.932716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:43.932717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:43.932995 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:43.933174 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.933415 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.933613 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:10:43.937517 27508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:10:43.944157 27508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:10:43.966531 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.956057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.10.181.129:33985)
0809 13:10:43.957983 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.957993 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.958023 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:10:43.958139 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.958147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.958402 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.958618 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:43.959307 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.960968 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.963892 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.965289 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.965529 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.965720 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.966024 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.966191 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1765,"thread_start_us":1601,"threads_started":1}
I20260809 13:10:43.966538 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.955891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39586)
0809 13:10:43.956683 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:43.956720 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:43.956900 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:43.958614 (+ 1714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:43.958622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:43.958658 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:43.959086 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.959101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.961164 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.961169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.963716 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.963731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.965566 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.965580 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.965775 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.965831 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 13:10:43.965962 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 13:10:43.966161 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":381,"threads_started":1}
I20260809 13:10:43.968286 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.957734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41557 (local address 127.0.0.1:34166)
0809 13:10:43.960314 (+ 2580us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:43.960330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:43.960436 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:43.961638 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:43.961642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:43.961667 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:43.962233 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.962245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.964018 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.964023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.965529 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.965545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.967541 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.967548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.967714 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.967745 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 13:10:43.967808 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 13:10:43.967924 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2470,"mutex_wait_us":3,"thread_start_us":1027,"threads_started":1}
I20260809 13:10:43.968930 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.958474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.10.181.131:41557)
0809 13:10:43.961033 (+ 2559us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.961041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.961073 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:10:43.961168 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.961175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.961409 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.961623 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:43.962437 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.963899 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.965713 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.967046 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.968222 (+ 1176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.968390 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.968456 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.968605 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2387,"thread_start_us":2221,"threads_started":1}
I20260809 13:10:43.969148 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.957144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:37933 (local address 127.0.0.1:33814)
0809 13:10:43.958259 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:43.958277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:43.958387 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:43.961928 (+ 3541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:43.961933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:43.961959 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:43.962476 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.962494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.964308 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.964313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.966395 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:43.966409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.968381 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.968387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:43.968552 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.968582 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:10:43.968670 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 13:10:43.968792 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":24,"thread_start_us":2924,"threads_started":1}
I20260809 13:10:43.969281 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:43.957407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.10.181.130:37933)
0809 13:10:43.961313 (+ 3906us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:43.961319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:43.961342 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:10:43.961413 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:43.961418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:43.961650 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:43.961838 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:43.962722 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.964157 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.966589 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:43.967894 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:43.968072 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:43.968232 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:43.968840 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 13:10:43.969007 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3758,"thread_start_us":3616,"threads_started":1}
I20260809 13:10:43.970992 27711 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb (DEFAULT_TABLE table=test-table [id=65413a0a4f4e4c898e0824517015b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:43.972788 27711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:43.972643 27928 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb (DEFAULT_TABLE table=test-table [id=65413a0a4f4e4c898e0824517015b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:43.972651 28143 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb (DEFAULT_TABLE table=test-table [id=65413a0a4f4e4c898e0824517015b8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:10:43.974032 27928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:43.974052 28143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:10:43.984082 28283 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Bootstrap starting.
I20260809 13:10:43.988071 28283 tablet_bootstrap.cc:654] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:43.988611 28285 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Bootstrap starting.
I20260809 13:10:43.989432 28283 log.cc:824] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:43.990039 28284 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Bootstrap starting.
I20260809 13:10:43.991986 28285 tablet_bootstrap.cc:654] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:43.992915 28284 tablet_bootstrap.cc:654] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Neither blocks nor log segments found. Creating new log.
I20260809 13:10:43.993223 28285 log.cc:824] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:43.993352 28283 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: No bootstrap required, opened a new log
I20260809 13:10:43.993637 28283 ts_tablet_manager.cc:1397] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 13:10:43.993924 28284 log.cc:824] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Log is configured to *not* fsync() on all Append() calls
I20260809 13:10:43.997267 28285 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: No bootstrap required, opened a new log
I20260809 13:10:43.997562 28285 ts_tablet_manager.cc:1397] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 13:10:43.999266 28284 tablet_bootstrap.cc:492] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: No bootstrap required, opened a new log
I20260809 13:10:43.999523 28284 ts_tablet_manager.cc:1397] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 13:10:44.003643 28283 raft_consensus.cc:348] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.004169 28283 raft_consensus.cc:374] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:44.004310 28283 raft_consensus.cc:729] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c875b93829f4e0d986fff79ad5559ee, State: Initialized, Role: FOLLOWER
I20260809 13:10:44.004982 28283 consensus_queue.cc:260] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.006716 28283 ts_tablet_manager.cc:1428] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 13:10:44.007781 28285 raft_consensus.cc:348] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.008090 28285 raft_consensus.cc:374] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:44.008184 28285 raft_consensus.cc:729] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bde286bc854e30b469ce059ebb5ebb, State: Initialized, Role: FOLLOWER
I20260809 13:10:44.008266 28284 raft_consensus.cc:348] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.008687 28284 raft_consensus.cc:374] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:10:44.008788 28284 raft_consensus.cc:729] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa5db7743e245068e93d459fcc93039, State: Initialized, Role: FOLLOWER
I20260809 13:10:44.008710 28285 consensus_queue.cc:260] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.009433 28284 consensus_queue.cc:260] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.009775 28256 heartbeater.cc:503] Master 127.10.181.190:33855 was elected leader, sending a full tablet report...
I20260809 13:10:44.011116 28037 heartbeater.cc:503] Master 127.10.181.190:33855 was elected leader, sending a full tablet report...
I20260809 13:10:44.011773 28284 ts_tablet_manager.cc:1428] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260809 13:10:44.012710 28285 ts_tablet_manager.cc:1428] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
W20260809 13:10:44.115924 27818 tablet.cc:2369] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:10:44.120414 28039 tablet.cc:2369] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:10:44.136611 28257 tablet.cc:2369] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:10:44.268340 28295 raft_consensus.cc:482] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:10:44.268639 28295 raft_consensus.cc:504] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.270355 28295 leader_election.cc:288] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c875b93829f4e0d986fff79ad5559ee (127.10.181.129:33985), 50bde286bc854e30b469ce059ebb5ebb (127.10.181.131:41557)
I20260809 13:10:44.279335 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.270921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:49345 (local address 127.10.181.129:33985)
0809 13:10:44.271134 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.271140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.271940 (+ 800us) server_negotiation.cc:407] Connection header received
0809 13:10:44.272109 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.272124 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.272200 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.272296 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:44.273081 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.274237 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.277110 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.278342 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.278489 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.278566 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.278936 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.279066 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:10:44.279331 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.270916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:49527 (local address 127.10.181.131:41557)
0809 13:10:44.271134 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.271140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.271908 (+ 768us) server_negotiation.cc:407] Connection header received
0809 13:10:44.272110 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.272115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.272200 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.272296 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:10:44.273160 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.274267 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.277098 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.278321 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.278489 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.278566 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.278936 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.279066 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:10:44.279567 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.270810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41557 (local address 127.10.181.130:49527)
0809 13:10:44.271778 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.271798 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:44.271975 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:44.272330 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:44.272335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:44.272382 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:44.273024 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:44.273035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.274371 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.274375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:44.276940 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:44.276958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.278577 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.278582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:44.278754 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.278808 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 13:10:44.278904 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 13:10:44.279083 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":167,"spinlock_wait_cycles":66432,"thread_start_us":511,"threads_started":1}
I20260809 13:10:44.279626 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.270843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.10.181.130:49345)
0809 13:10:44.271838 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.271851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 13:10:44.271970 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:10:44.272330 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:10:44.272335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:10:44.272385 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:10:44.272919 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:44.272939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.274369 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.274375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:44.276940 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:10:44.276958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.278555 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.278564 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:10:44.278749 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.278815 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 13:10:44.278904 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:10:44.279076 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":208,"spinlock_wait_cycles":36736,"thread_start_us":530,"threads_started":1}
I20260809 13:10:44.280776 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2c3efcd704a2ab6e6add29e8c9ecb" candidate_uuid: "8aa5db7743e245068e93d459fcc93039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c875b93829f4e0d986fff79ad5559ee" is_pre_election: true
I20260809 13:10:44.281335 27742 raft_consensus.cc:2454] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aa5db7743e245068e93d459fcc93039 in term 0.
I20260809 13:10:44.281601 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2c3efcd704a2ab6e6add29e8c9ecb" candidate_uuid: "8aa5db7743e245068e93d459fcc93039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bde286bc854e30b469ce059ebb5ebb" is_pre_election: true
I20260809 13:10:44.282181 27857 leader_election.cc:302] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c875b93829f4e0d986fff79ad5559ee, 8aa5db7743e245068e93d459fcc93039; no voters:
I20260809 13:10:44.282192 28182 raft_consensus.cc:2454] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aa5db7743e245068e93d459fcc93039 in term 0.
I20260809 13:10:44.282642 28295 raft_consensus.cc:2781] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:10:44.282806 28295 raft_consensus.cc:482] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:10:44.282920 28295 raft_consensus.cc:3037] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:44.285157 28295 raft_consensus.cc:504] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.285750 28295 leader_election.cc:288] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [CANDIDATE]: Term 1 election: Requested vote from peers 4c875b93829f4e0d986fff79ad5559ee (127.10.181.129:33985), 50bde286bc854e30b469ce059ebb5ebb (127.10.181.131:41557)
I20260809 13:10:44.286181 27742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2c3efcd704a2ab6e6add29e8c9ecb" candidate_uuid: "8aa5db7743e245068e93d459fcc93039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c875b93829f4e0d986fff79ad5559ee"
I20260809 13:10:44.286267 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2c3efcd704a2ab6e6add29e8c9ecb" candidate_uuid: "8aa5db7743e245068e93d459fcc93039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bde286bc854e30b469ce059ebb5ebb"
I20260809 13:10:44.286423 27742 raft_consensus.cc:3037] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:44.286453 28182 raft_consensus.cc:3037] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:10:44.289585 27742 raft_consensus.cc:2454] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aa5db7743e245068e93d459fcc93039 in term 1.
I20260809 13:10:44.289585 28182 raft_consensus.cc:2454] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aa5db7743e245068e93d459fcc93039 in term 1.
I20260809 13:10:44.290001 27857 leader_election.cc:302] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c875b93829f4e0d986fff79ad5559ee, 8aa5db7743e245068e93d459fcc93039; no voters:
I20260809 13:10:44.290294 28295 raft_consensus.cc:2781] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:10:44.290725 28295 raft_consensus.cc:686] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 1 LEADER]: Becoming Leader. State: Replica: 8aa5db7743e245068e93d459fcc93039, State: Running, Role: LEADER
I20260809 13:10:44.291361 28295 consensus_queue.cc:237] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } }
I20260809 13:10:44.296206 27509 catalog_manager.cc:5495] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aa5db7743e245068e93d459fcc93039 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "8aa5db7743e245068e93d459fcc93039" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8aa5db7743e245068e93d459fcc93039" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 37933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 } health_report { overall_health: UNKNOWN } } }
I20260809 13:10:44.329139 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.323977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.10.181.130:37933)
0809 13:10:44.324227 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.324231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.324270 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:10:44.324480 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.324484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.324565 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.324694 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:10:44.324997 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.325826 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.327162 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.327554 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.327651 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.328077 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:10:44.328614 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:10:44.328726 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.328797 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.328867 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:10:44.337492 28182 raft_consensus.cc:1261] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 1 FOLLOWER]: Refusing update from remote peer 8aa5db7743e245068e93d459fcc93039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:44.337494 27742 raft_consensus.cc:1261] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 1 FOLLOWER]: Refusing update from remote peer 8aa5db7743e245068e93d459fcc93039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:10:44.338660 28301 consensus_queue.cc:1035] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c875b93829f4e0d986fff79ad5559ee" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:44.339022 28295 consensus_queue.cc:1035] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50bde286bc854e30b469ce059ebb5ebb" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:10:44.350534 28306 mvcc.cc:205] Tried to move back new op lower bound from 7316607157595578368 to 7316607157421051904. Current Snapshot: MvccSnapshot[applied={T|T < 7316607157595578368}]
I20260809 13:10:44.354144 28312 mvcc.cc:205] Tried to move back new op lower bound from 7316607157595578368 to 7316607157421051904. Current Snapshot: MvccSnapshot[applied={T|T < 7316607157595578368}]
I20260809 13:10:44.363016 27509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:44.363313 27509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:10:44.367125 27509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4e53d396fc444c98fd1042761e7fb7d: Sending DeleteTablet for 3 replicas of tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb
I20260809 13:10:44.368358 27928 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:44 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33814
I20260809 13:10:44.368364 27711 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:44 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:39586
I20260809 13:10:44.368788 28143 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:10:44 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34166
I20260809 13:10:44.375376 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 27615
I20260809 13:10:44.378899 28324 tablet_replica.cc:330] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: stopping tablet replica
I20260809 13:10:44.379400 28324 raft_consensus.cc:2229] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:44.379942 28324 raft_consensus.cc:2258] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:44.384021 28324 ts_tablet_manager.cc:1905] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:44.391921 28324 ts_tablet_manager.cc:1918] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:44.392163 28324 log.cc:1196] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/8fe2c3efcd704a2ab6e6add29e8c9ecb
I20260809 13:10:44.392714 28324 ts_tablet_manager.cc:1939] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 50bde286bc854e30b469ce059ebb5ebb: Deleting consensus metadata
I20260809 13:10:44.393936 28323 tablet_replica.cc:330] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: stopping tablet replica
I20260809 13:10:44.394385 28323 raft_consensus.cc:2229] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:10:44.394797 27484 catalog_manager.cc:4841] TS 50bde286bc854e30b469ce059ebb5ebb (127.10.181.131:41557): tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb (table test-table [id=65413a0a4f4e4c898e0824517015b8ee]) successfully deleted
I20260809 13:10:44.395040 28323 raft_consensus.cc:2258] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 13:10:44 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:10:44.397517 28323 ts_tablet_manager.cc:1905] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:44.401968 28322 tablet_replica.cc:330] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: stopping tablet replica
I20260809 13:10:44.402382 28322 raft_consensus.cc:2229] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:10:44.402904 28322 raft_consensus.cc:2258] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:10:44.406117 28323 ts_tablet_manager.cc:1918] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:10:44.406337 28323 log.cc:1196] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/8fe2c3efcd704a2ab6e6add29e8c9ecb
I20260809 13:10:44.406478 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.378563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39588 (local address 127.10.181.129:33985)
0809 13:10:44.378767 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.378773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.382527 (+ 3754us) server_negotiation.cc:407] Connection header received
0809 13:10:44.383471 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.383477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.383562 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.383642 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:44.387198 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.388409 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.392075 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.392744 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.392848 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.398737 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:44.398754 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:44.398772 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:44.398817 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:44.403027 (+ 4210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.403887 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.403896 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.403899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.404065 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.404522 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.404528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.404531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.404979 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:44.405269 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.405883 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.406125 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:10:44.407276 28323 ts_tablet_manager.cc:1939] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 8aa5db7743e245068e93d459fcc93039: Deleting consensus metadata
I20260809 13:10:44.407795 27808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:44.410318 27488 catalog_manager.cc:4841] TS 8aa5db7743e245068e93d459fcc93039 (127.10.181.130:37933): tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb (table test-table [id=65413a0a4f4e4c898e0824517015b8ee]) successfully deleted
I20260809 13:10:44.408782 28322 ts_tablet_manager.cc:1905] T 8fe2c3efcd704a2ab6e6add29e8c9ecb P 4c875b93829f4e0d986fff79ad5559ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:10:44.496873 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 27615
W20260809 13:10:44.525101 27488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:33985 (error 108)
I20260809 13:10:44.525383 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 27826
I20260809 13:10:44.527921 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.526729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39590)
0809 13:10:44.527561 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.527703 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":564,"threads_started":1}
W20260809 13:10:44.528153 27488 catalog_manager.cc:4584] TS 4c875b93829f4e0d986fff79ad5559ee (127.10.181.129:33985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fe2c3efcd704a2ab6e6add29e8c9ecb: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Aug 09 13:10:44 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:10:44.542407 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.526280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.10.181.130:37933)
0809 13:10:44.526465 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.526471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.526559 (+ 88us) server_negotiation.cc:407] Connection header received
0809 13:10:44.526761 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.526768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.526904 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.527017 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:44.528585 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.529791 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.530973 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.531579 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.531706 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.535612 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:44.535631 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:44.535652 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:44.535706 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:44.539612 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.540253 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.540263 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.540267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.540445 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.540775 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.540781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.540784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.541249 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:44.541627 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.541778 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.541997 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:10:44.543223 28026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:44.581722 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.581199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39596)
0809 13:10:44.581404 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.581536 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 13:10:44.582844 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.582467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39612)
0809 13:10:44.582601 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.582686 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 13:10:44.643363 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.642879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39624)
0809 13:10:44.643106 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.643211 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 13:10:44.644371 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.643999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39626)
0809 13:10:44.644161 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.644232 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:10:44.656989 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 27826
I20260809 13:10:44.684360 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 28046
Aug 09 13:10:44 dist-test-slave-42z9 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:10:44.698730 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.685114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.10.181.131:41557)
0809 13:10:44.685303 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 13:10:44.685309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:10:44.685330 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:10:44.685525 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:10:44.685530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:10:44.685601 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:10:44.685689 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:10:44.687052 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.687815 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.689080 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:10:44.689515 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:10:44.689629 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:10:44.693751 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:10:44.693768 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:10:44.693785 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:10:44.693823 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:10:44.696456 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.697014 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.697022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.697024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.697179 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:10:44.697467 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:10:44.697471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:10:44.697474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:10:44.697833 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:10:44.698051 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 13:10:44.698317 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 13:10:44.698480 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:10:44.699436 28243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:44.725333 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.724727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39628)
0809 13:10:44.724992 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.725112 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 13:10:44.726509 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.726028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39632)
0809 13:10:44.726246 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.726336 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 13:10:44.788004 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 28046
I20260809 13:10:44.814845 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 27460
I20260809 13:10:44.816032 27564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:10:44.929240 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:10:44.928558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33985 (local address 127.0.0.1:39644)
0809 13:10:44.928805 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 13:10:44.928932 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:10:44.929504 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 27460
2026-08-09T13:10:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4604 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:10:45 dist-test-slave-42z9 krb5kdc[28340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:10:45 dist-test-slave-42z9 krb5kdc[28340](info): set up 2 sockets
Aug 09 13:10:45 dist-test-slave-42z9 krb5kdc[28340](info): commencing operation
krb5kdc: starting...
W20260809 13:10:46.162774 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:10:46 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:10:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangeradmin_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangeradmin_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangerlookup_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangerlookup_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
I20260809 13:10:46.532383 10966 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPW_N6O/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/postgres -l logfile start
2026-08-09 13:10:48.732 UTC [28385] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 13:10:48.732 UTC [28385] LOG: listening on IPv4 address "127.10.181.148", port 55003
2026-08-09 13:10:48.735 UTC [28385] LOG: listening on Unix socket "/tmp/.s.PGSQL.55003"
2026-08-09 13:10:48.757 UTC [28389] LOG: database system was shut down at 2026-08-09 13:10:47 UTC
2026-08-09 13:10:48.760 UTC [28385] LOG: database system is ready to accept connections
I20260809 13:10:49.852031 10966 mini_postgres.cc:96] Postgres bound to 55003
2026-08-09 13:10:49.876 UTC [28404] FATAL: database "slave" does not exist
127.10.181.148:55003 - accepting connections
I20260809 13:10:49.877341 10966 mini_ranger.cc:161] Starting Ranger...
I20260809 13:10:49.905494 10966 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 13:10:50.324100 10966 mini_ranger.cc:87] Created ranger Postgres database
I20260809 13:10:50.324221 10966 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin
2026-08-09 13:10:50,631 [I] DB FLAVOR :POSTGRES
2026-08-09 13:10:50,631 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 13:10:50,632 [I] Checking connection..
2026-08-09 13:10:50,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 13:10:50,880 [I] Checking connection passed.
2026-08-09 13:10:50,881 [I] --------- Verifying version history table ---------
2026-08-09 13:10:50,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:10:51,142 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 13:10:51,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:10:51,401 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 13:10:51,401 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 13:10:51,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 13:10:51.645 UTC [28512] WARNING: there is no transaction in progress
2026-08-09 13:10:51,652 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 13:10:51,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:10:51,899 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 13:10:51,899 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 13:10:51,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 13:10:52,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 13:10:52,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-42z9.c.gcp-upstream.internal','N') ;"
2026-08-09 13:10:52,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 13:10:52,873 [I] Table x_portal_user does not exist in database ranger
2026-08-09 13:10:52,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 13:10:53,125 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 13:10:53,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 13:10:53,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 13:10:53,635 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 13:10:53,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 13:10:54.508 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.523 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.538 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.551 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.566 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.582 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.643 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.650 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.664 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.673 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.682 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.696 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.705 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.711 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.717 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:54.723 UTC [28691] WARNING: there is no transaction in progress
2026-08-09 13:10:55,294 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 13:10:55,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 13:10:55,551 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 13:10:55,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 13:10:55,797 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 13:10:55,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 13:10:56,030 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 13:10:56,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 13:10:56,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 13:10:56,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-42z9.c.gcp-upstream.internal';"
2026-08-09 13:10:56,762 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 13:10:56,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 13:10:57,001 [I] DB_PATCHES have already been applied
I20260809 13:10:57.007166 10966 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPW_N6O/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 13:10:57.007256 10966 mini_ranger.cc:193] Using host: 127.10.181.148
I20260809 13:10:57.031520 10966 mini_ranger.cc:227] Ranger admin URL: http://127.10.181.148:52525
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 1:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 13:10:58.264367 10966 mini_ranger.cc:160] Time spent starting Ranger: real 8.387s user 0.007s sys 0.058s
Aug 09, 2026 1:11:17 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 1:11:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 1:11:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 1:11:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 1:11:18 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 13:11:19.036657 10966 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangerkms_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/rangerkms_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.181.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.181.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 13:11:19 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281079, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 13:11:19.247949 10966 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 13:11:19.273641 10966 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 13:11:20.052109 10966 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 13:11:20.052277 10966 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms
I20260809 13:11:20.052400 10966 mini_ranger_kms.cc:227] Using postgres at 127.10.181.148:55003
2026-08-09 13:11:20,727 [I] DB FLAVOR :POSTGRES
2026-08-09 13:11:20,727 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 13:11:20,727 [I] Checking connection
2026-08-09 13:11:20,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 13:11:20,986 [I] connection success
2026-08-09 13:11:20,986 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 13:11:20,986 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 13:11:20,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 13:11:21,224 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 13:11:21,224 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 13:11:21,224 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 13:11:21,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:55003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 13:11:21,477 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 13:11:21.962585 10966 mini_ranger_kms.cc:310] Created kms service
I20260809 13:11:22.171288 10966 mini_ranger_kms.cc:326] Created kudu user
I20260809 13:11:22.214175 10966 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 13:11:22 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281079, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:11:22.598762 10966 mini_ranger_kms.cc:384] Added ranger policy
I20260809 13:11:22.600257 10966 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 13:11:22.600343 10966 mini_ranger_kms.cc:240] Using host: 127.10.181.148
I20260809 13:11:22.628818 10966 mini_ranger_kms.cc:276] Ranger KMS PID: 28991
I20260809 13:11:22.629042 10966 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.181.148:47417
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 1:11:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 13:11:24 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786281084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:11:25.288360 10966 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.040s user 0.103s sys 0.450s
I20260809 13:11:25.288714 10966 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 13:11:25.288951 10966 mini_ranger_kms.cc:401] 127.10.181.148:47417/kms/v1/keys
Aug 09 13:11:25 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
Aug 09 13:11:25 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281084, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
Aug 09 13:11:27 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:11:28.013028 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43083
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:43083
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47417/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:11:28.373725 29072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:11:28.374008 29072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:11:28.374133 29072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:11:28.381237 29072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:11:28.381335 29072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:11:28.381408 29072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:11:28.381505 29072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:11:28.381559 29072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:11:28.391639 29072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.181.148:47417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:43083
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:11:28.392795 29072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:11:28.394068 29072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:11:28.405784 29078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:28.405831 29080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:28.405745 29086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:11:28.406652 29072 server_base.cc:1034] running on GCE node
I20260809 13:11:28.407325 29072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:11:28.408599 29072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:11:28.409865 29072 hybrid_clock.cc:648] HybridClock initialized: now 1786281088409751 us; error 32 us; skew 500 ppm
I20260809 13:11:28.410290 29072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:11:28 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:11:28.416313 29072 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:11:28.417779 29072 webserver.cc:460] Webserver started at http://127.10.181.190:45493/ using document root <none> and password file <none>
I20260809 13:11:28.418471 29072 fs_manager.cc:359] Metadata directory not provided
I20260809 13:11:28.418589 29072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:11:28.418888 29072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:11:28 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281088, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:11:28.471623 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "c98f2cf488e74a578cb4e113ae040c3b"
format_stamp: "Formatted at 2026-08-09 13:11:28 on dist-test-slave-42z9"
server_key: "ab7a2b91c7800cfb249ea57320338fae"
server_key_iv: "45e13b090253c24652a757cd769327ae"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:28.472601 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "c98f2cf488e74a578cb4e113ae040c3b"
format_stamp: "Formatted at 2026-08-09 13:11:28 on dist-test-slave-42z9"
server_key: "ab7a2b91c7800cfb249ea57320338fae"
server_key_iv: "45e13b090253c24652a757cd769327ae"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:28.478520 29072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:11:28.494896 29095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:28.496394 29072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:11:28.496572 29072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "c98f2cf488e74a578cb4e113ae040c3b"
format_stamp: "Formatted at 2026-08-09 13:11:28 on dist-test-slave-42z9"
server_key: "ab7a2b91c7800cfb249ea57320338fae"
server_key_iv: "45e13b090253c24652a757cd769327ae"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:28.496790 29072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:11:28.513427 29072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:11:28.516993 29072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:11:28.517280 29072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:11:28.536530 29072 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:43083
I20260809 13:11:28.536562 29184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:43083 every 8 connection(s)
I20260809 13:11:28.537878 29072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:11:28.541754 29185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:11:28.547991 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 29072
I20260809 13:11:28.548211 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:11:28.549044 10966 external_mini_cluster.cc:1428] Setting key 815001bbedaa26d10eb48f590a19a584
I20260809 13:11:28.553294 29185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: Bootstrap starting.
I20260809 13:11:28.556990 29185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: Neither blocks nor log segments found. Creating new log.
I20260809 13:11:28.558228 29185 log.cc:824] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:11:28.561175 29185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: No bootstrap required, opened a new log
Aug 09 13:11:28 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:11:28.570137 29185 raft_consensus.cc:348] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } }
I20260809 13:11:28.570592 29185 raft_consensus.cc:374] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:11:28.570713 29185 raft_consensus.cc:729] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98f2cf488e74a578cb4e113ae040c3b, State: Initialized, Role: FOLLOWER
I20260809 13:11:28.571246 29185 consensus_queue.cc:260] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } }
I20260809 13:11:28.571466 29185 raft_consensus.cc:388] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:11:28.571602 29185 raft_consensus.cc:482] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:11:28.571856 29190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:28.551177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.10.181.190:43083)
0809 13:11:28.552087 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:28.552099 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:28.552143 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:11:28.553932 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:28.553957 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:28.554319 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:28.554644 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:28.556215 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:28.557630 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:28.559307 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:28.559989 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:28.560130 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:28.564645 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:28.564653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:28.564667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:28.564711 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:28.567707 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:28.568500 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:28.568506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:28.568513 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:28.568675 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:28.569063 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:28.569067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:28.569069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:28.569645 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:28.569860 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:28.570337 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 13:11:28.570674 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":508,"threads_started":1}
I20260809 13:11:28.571878 29185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:11:28.574247 29185 raft_consensus.cc:504] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } }
I20260809 13:11:28.574791 29185 leader_election.cc:302] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c98f2cf488e74a578cb4e113ae040c3b; no voters:
I20260809 13:11:28.575330 29185 leader_election.cc:288] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:11:28.575443 29192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:11:28.576213 29192 raft_consensus.cc:686] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [term 1 LEADER]: Becoming Leader. State: Replica: c98f2cf488e74a578cb4e113ae040c3b, State: Running, Role: LEADER
I20260809 13:11:28.576697 29192 consensus_queue.cc:237] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } }
I20260809 13:11:28.577188 29185 sys_catalog.cc:564] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:11:28.579198 29193 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c98f2cf488e74a578cb4e113ae040c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } } }
I20260809 13:11:28.579978 29193 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [sys.catalog]: This master's current role is: LEADER
I20260809 13:11:28.579485 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c98f2cf488e74a578cb4e113ae040c3b. Latest consensus state: current_term: 1 leader_uuid: "c98f2cf488e74a578cb4e113ae040c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c98f2cf488e74a578cb4e113ae040c3b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43083 } } }
I20260809 13:11:28.580825 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b [sys.catalog]: This master's current role is: LEADER
I20260809 13:11:28.582275 29207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:11:28.589038 29207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:11:28.589640 29185 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 13:11:28.608700 29207 catalog_manager.cc:1347] Generated new cluster ID: 86e3ac5eb6dd438198df77c1a480cbbd
I20260809 13:11:28.608836 29207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:11:28.651021 29207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:11:28.653172 29207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:11:28.665640 29207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: Generated new TSK 0
I20260809 13:11:28.666863 29207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:11:29.398742 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:11:29 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 13:11:29.832404 29257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:11:29.832816 29257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:11:29.832993 29257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:11:29.844441 29257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:11:29.844599 29257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:11:29.844795 29257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:11:29.860420 29257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.181.148:47417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:11:29.862362 29257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:11:29.864315 29257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:11:29.882983 29274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:29.883427 29275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:29.884994 29278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:11:29.885572 29257 server_base.cc:1034] running on GCE node
I20260809 13:11:29.886261 29257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:11:29.893752 29257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:11:29.895016 29257 hybrid_clock.cc:648] HybridClock initialized: now 1786281089894959 us; error 60 us; skew 500 ppm
I20260809 13:11:29.895354 29257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:11:29 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:11:29.899657 29257 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:11:29.905890 29257 webserver.cc:460] Webserver started at http://127.10.181.129:41921/ using document root <none> and password file <none>
I20260809 13:11:29.906663 29257 fs_manager.cc:359] Metadata directory not provided
I20260809 13:11:29.906790 29257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:11:29.907109 29257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:11:29 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:11:29.935104 29257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "3470af820d5b4930bdf7e4f864dcfc9b"
format_stamp: "Formatted at 2026-08-09 13:11:29 on dist-test-slave-42z9"
server_key: "478c117e27e2007916a2fbe2e33c43fc"
server_key_iv: "93507882f2778fdb8223cfb06804d509"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:29.936333 29257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "3470af820d5b4930bdf7e4f864dcfc9b"
format_stamp: "Formatted at 2026-08-09 13:11:29 on dist-test-slave-42z9"
server_key: "478c117e27e2007916a2fbe2e33c43fc"
server_key_iv: "93507882f2778fdb8223cfb06804d509"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:29.944535 29257 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260809 13:11:30.000546 29286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:30.002698 29257 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.001s
I20260809 13:11:30.002924 29257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "3470af820d5b4930bdf7e4f864dcfc9b"
format_stamp: "Formatted at 2026-08-09 13:11:29 on dist-test-slave-42z9"
server_key: "478c117e27e2007916a2fbe2e33c43fc"
server_key_iv: "93507882f2778fdb8223cfb06804d509"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:30.003212 29257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:11:30.042773 29257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:11:30.048137 29257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:11:30.048478 29257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:11:30.053052 29257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:11:30.055234 29257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:11:30.055393 29257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 13:11:30.055608 29257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:11:30.055737 29257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:30.136356 29257 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:37669
I20260809 13:11:30.136417 29452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:37669 every 8 connection(s)
I20260809 13:11:30.138221 29257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:11:30.139822 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 29257
I20260809 13:11:30.140034 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:11:30.140810 10966 external_mini_cluster.cc:1428] Setting key 6da63b540dc82a533c88d1c8c91669d6
Aug 09 13:11:30 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:11:30.190862 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:30.143845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:45889 (local address 127.10.181.190:43083)
0809 13:11:30.165315 (+ 21470us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:30.165322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:30.169399 (+ 4077us) server_negotiation.cc:407] Connection header received
0809 13:11:30.169516 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:30.169521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:30.169627 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:30.169835 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:30.173561 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:30.175025 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:30.177780 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:30.178444 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:30.178550 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:30.183361 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:30.183369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:30.183374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:30.183419 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:30.186966 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:30.187900 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:30.187906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:30.187910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:30.188073 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:30.188636 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:30.188642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:30.188645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:30.189017 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:30.189150 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:30.190211 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0809 13:11:30.190464 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21327,"thread_start_us":21481,"threads_started":1}
I20260809 13:11:30.193107 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:30.142829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43083 (local address 127.10.181.129:45889)
0809 13:11:30.166079 (+ 23250us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:30.166154 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:30.168927 (+ 2773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:30.171137 (+ 2210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:30.171151 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:30.171917 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:11:30.172773 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:30.172798 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:30.175254 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:30.175260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:30.177558 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:30.177634 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:30.178973 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:30.178981 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:30.179272 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:30.180118 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:11:30.180150 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:11:30.182988 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:11:30.187142 (+ 4154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:30.187153 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:30.187171 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:30.187756 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:30.188226 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:30.188230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:30.188234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:30.188503 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:30.189247 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:11:30.189255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:11:30.189778 (+ 523us) client_negotiation.cc:769] Sending connection context
0809 13:11:30.191166 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0809 13:11:30.191733 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22884,"thread_start_us":27041,"threads_started":1}
I20260809 13:11:30.195223 29453 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43083
I20260809 13:11:30.195554 29453 heartbeater.cc:464] Registering TS with master...
I20260809 13:11:30.196367 29453 heartbeater.cc:511] Master 127.10.181.190:43083 requested a full tablet report, sending...
I20260809 13:11:30.199023 29128 ts_manager.cc:194] Registered new tserver with Master: 3470af820d5b4930bdf7e4f864dcfc9b (127.10.181.129:37669)
I20260809 13:11:30.201174 29128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:45889
Aug 09 13:11:30 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
W20260809 13:11:30.266713 10966 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.181.130: real 0.121s user 0.010s sys 0.016s
I20260809 13:11:30.267776 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:11:30 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281089, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
W20260809 13:11:30.776846 29460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:11:30.777336 29460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:11:30.777544 29460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:11:30.789104 29460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:11:30.789331 29460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:11:30.789602 29460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:11:30.805109 29460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.181.148:47417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:11:30.807109 29460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:11:30.808984 29460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:11:30.832297 29472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:30.832266 29475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:30.833119 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:11:30.833380 29460 server_base.cc:1034] running on GCE node
I20260809 13:11:30.834225 29460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:11:30.835345 29460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:11:30.836563 29460 hybrid_clock.cc:648] HybridClock initialized: now 1786281090836502 us; error 61 us; skew 500 ppm
I20260809 13:11:30.836874 29460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:11:30 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281090, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:11:30.840094 29460 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:11:30.842521 29460 webserver.cc:460] Webserver started at http://127.10.181.130:40369/ using document root <none> and password file <none>
I20260809 13:11:30.843199 29460 fs_manager.cc:359] Metadata directory not provided
I20260809 13:11:30.843309 29460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:11:30.843611 29460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:11:30 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281090, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:11:30.859153 29460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "4bd2c20a2b144031815bd5b56887c131"
format_stamp: "Formatted at 2026-08-09 13:11:30 on dist-test-slave-42z9"
server_key: "579f7a675fb65d0bd6898a5ccf7b6ce7"
server_key_iv: "c2e52b3ead890c9ff260a95369690bbf"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:30.859985 29460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "4bd2c20a2b144031815bd5b56887c131"
format_stamp: "Formatted at 2026-08-09 13:11:30 on dist-test-slave-42z9"
server_key: "579f7a675fb65d0bd6898a5ccf7b6ce7"
server_key_iv: "c2e52b3ead890c9ff260a95369690bbf"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:30.865432 29460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 13:11:30.882995 29495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:30.884984 29460 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 13:11:30.885185 29460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "4bd2c20a2b144031815bd5b56887c131"
format_stamp: "Formatted at 2026-08-09 13:11:30 on dist-test-slave-42z9"
server_key: "579f7a675fb65d0bd6898a5ccf7b6ce7"
server_key_iv: "c2e52b3ead890c9ff260a95369690bbf"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:30.885478 29460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:11:30.916455 29460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:11:30.921650 29460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:11:30.921972 29460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:11:30.923575 29460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:11:30.925522 29460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:11:30.925614 29460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:30.925738 29460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:11:30.925796 29460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:30.978338 29460 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:46485
I20260809 13:11:30.978360 29687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:46485 every 8 connection(s)
I20260809 13:11:30.980288 29460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:11:30.985095 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 29460
I20260809 13:11:30.985275 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:11:30.985857 10966 external_mini_cluster.cc:1428] Setting key 7db5504d759c7721fca3a076e55146cd
Aug 09 13:11:31 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281090, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:11:31.017261 29694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:30.984586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:38433 (local address 127.10.181.190:43083)
0809 13:11:30.997999 (+ 13413us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:30.998006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:30.998658 (+ 652us) server_negotiation.cc:407] Connection header received
0809 13:11:31.003266 (+ 4608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.003272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.003392 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.003498 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:31.004969 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.005757 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.007285 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.007752 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.008347 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.011292 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:31.011297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:31.011300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:31.011333 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:31.014279 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.014983 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.014987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.014989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.015110 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.015536 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.015540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.015542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.015821 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:31.015954 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.016647 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.016876 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13165,"thread_start_us":13046,"threads_started":1}
I20260809 13:11:31.018338 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:30.983954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43083 (local address 127.10.181.130:38433)
0809 13:11:30.998477 (+ 14523us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:30.998524 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.002989 (+ 4465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.003748 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.003757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.004199 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:11:31.004797 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.004816 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.005911 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.005915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.007141 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.007154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.007975 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.007983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.008288 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.008990 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:11:31.009013 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:11:31.011089 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:11:31.014414 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:31.014423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:31.014439 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:31.014867 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:31.015213 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:31.015219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:31.015223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:31.015418 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:31.015955 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:11:31.015963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:11:31.016278 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.016521 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.017017 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14271,"thread_start_us":14876,"threads_started":1}
I20260809 13:11:31.019714 29688 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43083
I20260809 13:11:31.020012 29688 heartbeater.cc:464] Registering TS with master...
I20260809 13:11:31.020889 29688 heartbeater.cc:511] Master 127.10.181.190:43083 requested a full tablet report, sending...
I20260809 13:11:31.022303 29128 ts_manager.cc:194] Registered new tserver with Master: 4bd2c20a2b144031815bd5b56887c131 (127.10.181.130:46485)
I20260809 13:11:31.023298 29128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:38433
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:11:31.073083 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:11:31.204842 29453 heartbeater.cc:503] Master 127.10.181.190:43083 was elected leader, sending a full tablet report...
W20260809 13:11:31.432753 29697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:11:31.433022 29697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:11:31.433161 29697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:11:31.440518 29697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:11:31.440616 29697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:11:31.440778 29697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:11:31.450466 29697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.181.148:47417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:43083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:11:31.451761 29697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:11:31.453047 29697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:11:31.465794 29705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:31.465756 29710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:11:31.465834 29707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:11:31.466915 29697 server_base.cc:1034] running on GCE node
I20260809 13:11:31.467440 29697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:11:31.468287 29697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:11:31.469508 29697 hybrid_clock.cc:648] HybridClock initialized: now 1786281091469433 us; error 57 us; skew 500 ppm
I20260809 13:11:31.469810 29697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:11:31 dist-test-slave-42z9 krb5kdc[28340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281091, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:11:31.472808 29697 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:11:31.474215 29697 webserver.cc:460] Webserver started at http://127.10.181.131:44449/ using document root <none> and password file <none>
I20260809 13:11:31.474846 29697 fs_manager.cc:359] Metadata directory not provided
I20260809 13:11:31.474972 29697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:11:31.475261 29697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:11:31 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281091, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:11:31.490962 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "d4048851c5d24901b440bd34e5dea0b1"
format_stamp: "Formatted at 2026-08-09 13:11:31 on dist-test-slave-42z9"
server_key: "c31bbc89c9212af038b058f6dc58827f"
server_key_iv: "4fb31beb6832474853196ecb1e101fbd"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:31.491823 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "d4048851c5d24901b440bd34e5dea0b1"
format_stamp: "Formatted at 2026-08-09 13:11:31 on dist-test-slave-42z9"
server_key: "c31bbc89c9212af038b058f6dc58827f"
server_key_iv: "4fb31beb6832474853196ecb1e101fbd"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:31.497334 29697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:11:31.514336 29721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:31.516199 29697 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:11:31.516434 29697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "d4048851c5d24901b440bd34e5dea0b1"
format_stamp: "Formatted at 2026-08-09 13:11:31 on dist-test-slave-42z9"
server_key: "c31bbc89c9212af038b058f6dc58827f"
server_key_iv: "4fb31beb6832474853196ecb1e101fbd"
server_key_version: "kuduclusterkey@0"
I20260809 13:11:31.516741 29697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:11:31.530280 29697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:11:31.533951 29697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:11:31.534224 29697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:11:31.535102 29697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:11:31.536415 29697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:11:31.536494 29697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:31.536612 29697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:11:31.536669 29697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:11:31.568282 29697 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:46333
I20260809 13:11:31.568320 29900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:46333 every 8 connection(s)
I20260809 13:11:31.569540 29697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:11:31.575271 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 29697
I20260809 13:11:31.575501 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:11:31.575956 10966 external_mini_cluster.cc:1428] Setting key e93196a3e30b00da129a72dcf672a855
Aug 09 13:11:31 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281091, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:11:31.588734 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.571946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:43781 (local address 127.10.181.190:43083)
0809 13:11:31.572443 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.572449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.573419 (+ 970us) server_negotiation.cc:407] Connection header received
0809 13:11:31.575219 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.575225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.575325 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.575463 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:31.576940 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.577807 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.579715 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.580171 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.580259 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.583528 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:31.583533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:31.583537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:31.583570 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:31.586141 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.586805 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.586809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.586811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.586931 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.587325 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.587328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.587330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.587581 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:31.587695 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.588303 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.588461 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":200,"threads_started":1}
I20260809 13:11:31.589805 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.572532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43083 (local address 127.10.181.131:43781)
0809 13:11:31.573254 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.573292 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.574922 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.575686 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.575694 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.576148 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:11:31.576733 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.576753 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.578015 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.578021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.579582 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.579595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.580391 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.580396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.580624 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.581220 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:11:31.581242 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:11:31.583340 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:11:31.586266 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:31.586274 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:31.586285 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:31.586716 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:31.587027 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:11:31.587030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:11:31.587033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:11:31.587243 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:11:31.587702 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:11:31.587708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:11:31.588007 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.588234 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.588627 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":282,"threads_started":1}
I20260809 13:11:31.591090 29901 heartbeater.cc:347] Connected to a master server at 127.10.181.190:43083
I20260809 13:11:31.591444 29901 heartbeater.cc:464] Registering TS with master...
I20260809 13:11:31.592231 29901 heartbeater.cc:511] Master 127.10.181.190:43083 requested a full tablet report, sending...
I20260809 13:11:31.593705 29128 ts_manager.cc:194] Registered new tserver with Master: d4048851c5d24901b440bd34e5dea0b1 (127.10.181.131:46333)
I20260809 13:11:31.594517 29128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:43781
I20260809 13:11:31.603386 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:11:31.620301 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.608003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.10.181.190:43083)
0809 13:11:31.608191 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.608197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.608531 (+ 334us) server_negotiation.cc:407] Connection header received
0809 13:11:31.608706 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.608710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.608784 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.608863 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:31.610480 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.611183 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.612479 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.612899 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.613008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.615355 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:31.615360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:31.615364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:31.615394 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:31.617696 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.618325 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.618329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.618332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.618453 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.618822 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.618827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.618830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.619107 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:31.619351 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.619702 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.619990 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:11:31.624416 29128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:11:31.631597 29128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:11:31.654510 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.646729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.10.181.129:37669)
0809 13:11:31.647793 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.647802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.647827 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:11:31.647915 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.647922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.648129 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.648312 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:11:31.649159 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.650286 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.652427 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.653393 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.653608 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.653799 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.654029 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.654198 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":725,"threads_started":1}
I20260809 13:11:31.654707 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.646194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:37669 (local address 127.0.0.1:58524)
0809 13:11:31.646811 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.646827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.646934 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.648559 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.648564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.648588 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:11:31.648992 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.649005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.650435 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.650440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.652244 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.652258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.653606 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.653612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.653809 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.653860 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.653972 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.654147 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1,"thread_start_us":683,"threads_started":1}
I20260809 13:11:31.655406 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.644419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:46485 (local address 127.0.0.1:57972)
0809 13:11:31.645761 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.645783 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.645898 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.647278 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.647287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.647314 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:11:31.647803 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.647817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.649514 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.649519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.652454 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.652463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.654549 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.654558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.654726 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.654759 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.654844 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.654982 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"mutex_wait_us":189,"thread_start_us":1076,"threads_started":1}
I20260809 13:11:31.655427 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.645071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.10.181.130:46485)
0809 13:11:31.646362 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.646371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.646401 (+ 30us) server_negotiation.cc:407] Connection header received
0809 13:11:31.646510 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.646517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.646785 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.647012 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:11:31.647997 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.649384 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.652628 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.654057 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.654200 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.654364 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.654997 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.655157 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1131,"thread_start_us":1043,"threads_started":1}
I20260809 13:11:31.655629 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.644465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.10.181.131:46333)
0809 13:11:31.646497 (+ 2032us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.646506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.646542 (+ 36us) server_negotiation.cc:407] Connection header received
0809 13:11:31.646642 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.646648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.646893 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.647078 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:11:31.647873 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.649408 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.652635 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.654057 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.654222 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.654363 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.654982 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.655147 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1865,"thread_start_us":1826,"threads_started":1}
I20260809 13:11:31.656383 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.644166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:46333 (local address 127.0.0.1:41062)
0809 13:11:31.645120 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.645171 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.645349 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.647084 (+ 1735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.647090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.647124 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:11:31.647644 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.647664 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.649571 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.649575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.652446 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.652462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.654636 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.654642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.654799 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.654827 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.655634 (+ 807us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.656057 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":243,"thread_start_us":531,"threads_started":1}
I20260809 13:11:31.659194 29591 tablet_service.cc:1467] Processing CreateTablet for tablet 86566644f50a48aba310e7d943b28859 (DEFAULT_TABLE table=test-table [id=3d98e2a0f0e84beba62279f75df8a75d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:11:31.659212 29361 tablet_service.cc:1467] Processing CreateTablet for tablet 86566644f50a48aba310e7d943b28859 (DEFAULT_TABLE table=test-table [id=3d98e2a0f0e84beba62279f75df8a75d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:11:31.659838 29797 tablet_service.cc:1467] Processing CreateTablet for tablet 86566644f50a48aba310e7d943b28859 (DEFAULT_TABLE table=test-table [id=3d98e2a0f0e84beba62279f75df8a75d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:11:31.660475 29591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86566644f50a48aba310e7d943b28859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:11:31.661233 29797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86566644f50a48aba310e7d943b28859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:11:31.661475 29361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86566644f50a48aba310e7d943b28859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:11:31.671484 29925 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Bootstrap starting.
I20260809 13:11:31.675381 29925 tablet_bootstrap.cc:654] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Neither blocks nor log segments found. Creating new log.
I20260809 13:11:31.676420 29925 log.cc:824] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Log is configured to *not* fsync() on all Append() calls
I20260809 13:11:31.677352 29926 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Bootstrap starting.
I20260809 13:11:31.678234 29927 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Bootstrap starting.
I20260809 13:11:31.681008 29925 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: No bootstrap required, opened a new log
I20260809 13:11:31.681317 29925 ts_tablet_manager.cc:1397] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 13:11:31.682112 29926 tablet_bootstrap.cc:654] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Neither blocks nor log segments found. Creating new log.
I20260809 13:11:31.683454 29926 log.cc:824] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:11:31.684657 29927 tablet_bootstrap.cc:654] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Neither blocks nor log segments found. Creating new log.
I20260809 13:11:31.685688 29927 log.cc:824] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Log is configured to *not* fsync() on all Append() calls
I20260809 13:11:31.686719 29926 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: No bootstrap required, opened a new log
I20260809 13:11:31.686995 29926 ts_tablet_manager.cc:1397] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 13:11:31.688086 29927 tablet_bootstrap.cc:492] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: No bootstrap required, opened a new log
I20260809 13:11:31.688316 29927 ts_tablet_manager.cc:1397] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260809 13:11:31.691778 29925 raft_consensus.cc:348] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.692237 29925 raft_consensus.cc:374] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:11:31.692338 29925 raft_consensus.cc:729] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bd2c20a2b144031815bd5b56887c131, State: Initialized, Role: FOLLOWER
I20260809 13:11:31.692958 29925 consensus_queue.cc:260] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.697104 29927 raft_consensus.cc:348] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.697466 29927 raft_consensus.cc:374] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:11:31.697623 29927 raft_consensus.cc:729] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4048851c5d24901b440bd34e5dea0b1, State: Initialized, Role: FOLLOWER
I20260809 13:11:31.698212 29688 heartbeater.cc:503] Master 127.10.181.190:43083 was elected leader, sending a full tablet report...
I20260809 13:11:31.698340 29927 consensus_queue.cc:260] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.698951 29925 ts_tablet_manager.cc:1428] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Time spent starting tablet: real 0.018s user 0.007s sys 0.011s
I20260809 13:11:31.699460 29901 heartbeater.cc:503] Master 127.10.181.190:43083 was elected leader, sending a full tablet report...
I20260809 13:11:31.697655 29926 raft_consensus.cc:348] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.699683 29926 raft_consensus.cc:374] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:11:31.699785 29926 raft_consensus.cc:729] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3470af820d5b4930bdf7e4f864dcfc9b, State: Initialized, Role: FOLLOWER
I20260809 13:11:31.699972 29927 ts_tablet_manager.cc:1428] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260809 13:11:31.700456 29926 consensus_queue.cc:260] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.702260 29926 ts_tablet_manager.cc:1428] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 13:11:31.748778 29691 tablet.cc:2369] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:11:31.809619 29933 raft_consensus.cc:482] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:11:31.810026 29933 raft_consensus.cc:504] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.811841 29933 leader_election.cc:288] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bd2c20a2b144031815bd5b56887c131 (127.10.181.130:46485), 3470af820d5b4930bdf7e4f864dcfc9b (127.10.181.129:37669)
I20260809 13:11:31.818468 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.812164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:37669 (local address 127.10.181.131:60661)
0809 13:11:31.812471 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.812492 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.812622 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.813029 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.813034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.813069 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:11:31.813281 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.813293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.814485 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.814489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.816439 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.816456 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.817824 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.817830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.817984 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.818010 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.818099 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.818198 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1}
I20260809 13:11:31.818512 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.812469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60661 (local address 127.10.181.129:37669)
0809 13:11:31.812679 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.812685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.812708 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:11:31.812782 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.812788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.812863 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.812968 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:11:31.813414 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.814351 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.816605 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.817657 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.817817 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.817892 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.818156 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.818278 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:11:31.819257 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.812268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:36847 (local address 127.10.181.130:46485)
0809 13:11:31.812516 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.812521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.813194 (+ 673us) server_negotiation.cc:407] Connection header received
0809 13:11:31.813365 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.813369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.813477 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.813573 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:11:31.814106 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.814890 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.817076 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.818202 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.818343 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.818411 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.818889 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.819016 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 13:11:31.819327 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.812251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:46485 (local address 127.10.181.131:36847)
0809 13:11:31.813078 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 13:11:31.813094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:11:31.813235 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:11:31.813621 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:11:31.813624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:11:31.813651 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:11:31.813955 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.813964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.815039 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.815044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.816925 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:11:31.816937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.818454 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.818461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:11:31.818627 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.818660 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:11:31.818837 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 13:11:31.818962 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":26,"thread_start_us":425,"threads_started":1}
I20260809 13:11:31.819448 29390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86566644f50a48aba310e7d943b28859" candidate_uuid: "d4048851c5d24901b440bd34e5dea0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" is_pre_election: true
I20260809 13:11:31.819976 29390 raft_consensus.cc:2454] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4048851c5d24901b440bd34e5dea0b1 in term 0.
I20260809 13:11:31.820286 29622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86566644f50a48aba310e7d943b28859" candidate_uuid: "d4048851c5d24901b440bd34e5dea0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bd2c20a2b144031815bd5b56887c131" is_pre_election: true
I20260809 13:11:31.820690 29729 leader_election.cc:302] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3470af820d5b4930bdf7e4f864dcfc9b, d4048851c5d24901b440bd34e5dea0b1; no voters:
I20260809 13:11:31.820868 29622 raft_consensus.cc:2454] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4048851c5d24901b440bd34e5dea0b1 in term 0.
W20260809 13:11:31.821066 29903 tablet.cc:2369] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:11:31.821201 29933 raft_consensus.cc:2781] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:11:31.821404 29933 raft_consensus.cc:482] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:11:31.821545 29933 raft_consensus.cc:3037] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:11:31.823793 29933 raft_consensus.cc:504] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.824391 29933 leader_election.cc:288] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [CANDIDATE]: Term 1 election: Requested vote from peers 4bd2c20a2b144031815bd5b56887c131 (127.10.181.130:46485), 3470af820d5b4930bdf7e4f864dcfc9b (127.10.181.129:37669)
I20260809 13:11:31.824855 29622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86566644f50a48aba310e7d943b28859" candidate_uuid: "d4048851c5d24901b440bd34e5dea0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bd2c20a2b144031815bd5b56887c131"
I20260809 13:11:31.824856 29390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86566644f50a48aba310e7d943b28859" candidate_uuid: "d4048851c5d24901b440bd34e5dea0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3470af820d5b4930bdf7e4f864dcfc9b"
I20260809 13:11:31.825042 29390 raft_consensus.cc:3037] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:11:31.825037 29622 raft_consensus.cc:3037] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:11:31.827411 29390 raft_consensus.cc:2454] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4048851c5d24901b440bd34e5dea0b1 in term 1.
I20260809 13:11:31.827437 29622 raft_consensus.cc:2454] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4048851c5d24901b440bd34e5dea0b1 in term 1.
I20260809 13:11:31.827782 29723 leader_election.cc:302] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bd2c20a2b144031815bd5b56887c131, d4048851c5d24901b440bd34e5dea0b1; no voters:
I20260809 13:11:31.828096 29933 raft_consensus.cc:2781] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:11:31.828521 29933 raft_consensus.cc:686] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 1 LEADER]: Becoming Leader. State: Replica: d4048851c5d24901b440bd34e5dea0b1, State: Running, Role: LEADER
I20260809 13:11:31.829008 29933 consensus_queue.cc:237] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } }
I20260809 13:11:31.833528 29124 catalog_manager.cc:5495] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4048851c5d24901b440bd34e5dea0b1 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "d4048851c5d24901b440bd34e5dea0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4048851c5d24901b440bd34e5dea0b1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 46333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 } health_report { overall_health: UNKNOWN } } }
I20260809 13:11:31.891348 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.886185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.10.181.131:46333)
0809 13:11:31.886350 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.886354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.886371 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:11:31.886533 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.886537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.886629 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.886750 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:11:31.887095 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.887842 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.889315 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.889729 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.889818 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.890260 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:11:31.890874 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:11:31.890999 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.891084 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.891150 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 13:11:31.899794 29622 raft_consensus.cc:1261] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 1 FOLLOWER]: Refusing update from remote peer d4048851c5d24901b440bd34e5dea0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:11:31.899793 29390 raft_consensus.cc:1261] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 1 FOLLOWER]: Refusing update from remote peer d4048851c5d24901b440bd34e5dea0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:11:31.900880 29933 consensus_queue.cc:1035] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3470af820d5b4930bdf7e4f864dcfc9b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:11:31.901228 29936 consensus_queue.cc:1035] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bd2c20a2b144031815bd5b56887c131" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:11:31.914397 29939 mvcc.cc:205] Tried to move back new op lower bound from 7316607352410152960 to 7316607352135475200. Current Snapshot: MvccSnapshot[applied={T|T < 7316607352410152960}]
I20260809 13:11:31.929798 29124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:11:31.930051 29124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:11:31.933343 29124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c98f2cf488e74a578cb4e113ae040c3b: Sending DeleteTablet for 3 replicas of tablet 86566644f50a48aba310e7d943b28859
I20260809 13:11:31.934459 29797 tablet_service.cc:1514] Processing DeleteTablet for tablet 86566644f50a48aba310e7d943b28859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:11:31 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:41062
I20260809 13:11:31.934808 29591 tablet_service.cc:1514] Processing DeleteTablet for tablet 86566644f50a48aba310e7d943b28859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:11:31 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57972
I20260809 13:11:31.935369 29957 tablet_replica.cc:330] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: stopping tablet replica
I20260809 13:11:31.935832 29957 raft_consensus.cc:2229] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:11:31.935999 29361 tablet_service.cc:1514] Processing DeleteTablet for tablet 86566644f50a48aba310e7d943b28859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:11:31 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:58524
I20260809 13:11:31.936545 29957 raft_consensus.cc:2258] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:11:31.937861 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 29257
I20260809 13:11:31.938616 29957 ts_tablet_manager.cc:1905] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:11:31.946398 29957 ts_tablet_manager.cc:1918] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:11:31.946641 29957 log.cc:1196] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/86566644f50a48aba310e7d943b28859
I20260809 13:11:31.947258 29957 ts_tablet_manager.cc:1939] T 86566644f50a48aba310e7d943b28859 P d4048851c5d24901b440bd34e5dea0b1: Deleting consensus metadata
I20260809 13:11:31.949074 29101 catalog_manager.cc:4841] TS d4048851c5d24901b440bd34e5dea0b1 (127.10.181.131:46333): tablet 86566644f50a48aba310e7d943b28859 (table test-table [id=3d98e2a0f0e84beba62279f75df8a75d]) successfully deleted
I20260809 13:11:31.950001 29959 tablet_replica.cc:330] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: stopping tablet replica
I20260809 13:11:31.950534 29959 raft_consensus.cc:2229] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:11:31.951152 29959 raft_consensus.cc:2258] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:11:31.952467 29962 tablet_replica.cc:330] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: stopping tablet replica
I20260809 13:11:31.952978 29962 raft_consensus.cc:2229] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:11:31.953552 29962 raft_consensus.cc:2258] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:11:31.954025 29959 ts_tablet_manager.cc:1905] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:11:31.955471 29962 ts_tablet_manager.cc:1905] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 13:11:31 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:11:31.960803 29959 ts_tablet_manager.cc:1918] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:11:31.960989 29959 log.cc:1196] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/86566644f50a48aba310e7d943b28859
I20260809 13:11:31.961562 29959 ts_tablet_manager.cc:1939] T 86566644f50a48aba310e7d943b28859 P 4bd2c20a2b144031815bd5b56887c131: Deleting consensus metadata
I20260809 13:11:31.962512 29962 ts_tablet_manager.cc:1918] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:11:31.962725 29962 log.cc:1196] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/86566644f50a48aba310e7d943b28859
I20260809 13:11:31.963416 29098 catalog_manager.cc:4841] TS 4bd2c20a2b144031815bd5b56887c131 (127.10.181.130:46485): tablet 86566644f50a48aba310e7d943b28859 (table test-table [id=3d98e2a0f0e84beba62279f75df8a75d]) successfully deleted
I20260809 13:11:31.963658 29962 ts_tablet_manager.cc:1939] T 86566644f50a48aba310e7d943b28859 P 3470af820d5b4930bdf7e4f864dcfc9b: Deleting consensus metadata
I20260809 13:11:31.965236 29101 catalog_manager.cc:4841] TS 3470af820d5b4930bdf7e4f864dcfc9b (127.10.181.129:37669): tablet 86566644f50a48aba310e7d943b28859 (table test-table [id=3d98e2a0f0e84beba62279f75df8a75d]) successfully deleted
I20260809 13:11:31.966646 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:31.939519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.10.181.129:37669)
0809 13:11:31.939698 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:31.939704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:31.943499 (+ 3795us) server_negotiation.cc:407] Connection header received
0809 13:11:31.944321 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:31.944327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:31.944418 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:31.944501 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:31.947944 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.949127 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.951233 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:31.951934 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:31.952037 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:31.958191 (+ 6154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:31.958206 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:31.958223 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:31.958270 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:31.962414 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.964237 (+ 1823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.964246 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.964250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.964432 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:31.964933 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:31.964938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:31.964941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:31.965343 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:31.965693 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:31.966064 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 13:11:31.966317 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:11:31.967350 29448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:11:32.057324 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 29257
I20260809 13:11:32.082967 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 29460
Aug 09 13:11:32 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:11:32.099553 29922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:32.083841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.10.181.130:46485)
0809 13:11:32.084022 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:32.084027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:32.084118 (+ 91us) server_negotiation.cc:407] Connection header received
0809 13:11:32.084311 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:32.084317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:32.084399 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:32.084500 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:32.085804 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:32.086560 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:32.087899 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:32.088281 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:32.088401 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:32.093000 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:32.093019 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:32.093042 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:32.093086 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:32.096771 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:32.097399 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:32.097408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:32.097412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:32.097598 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:32.097922 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:32.097926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:32.097930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:32.098402 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:32.098705 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:32.098930 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 13:11:32.099159 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:11:32.100274 29683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:11:32.214589 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 29460
I20260809 13:11:32.237043 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 29697
Aug 09 13:11:32 dist-test-slave-42z9 krb5kdc[28340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:11:32.253324 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:11:32.237922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.10.181.131:46333)
0809 13:11:32.238110 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:11:32.238117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:11:32.238208 (+ 91us) server_negotiation.cc:407] Connection header received
0809 13:11:32.238440 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:11:32.238445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:11:32.238527 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:11:32.238615 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:11:32.239946 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:32.240753 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:32.242090 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:11:32.242528 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:11:32.242643 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:11:32.247316 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:11:32.247329 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:11:32.247346 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:11:32.247400 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:11:32.250414 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:32.251266 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:32.251277 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:32.251279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:32.251458 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:11:32.251844 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:11:32.251847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:11:32.251849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:11:32.252192 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:11:32.252444 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 13:11:32.252848 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 13:11:32.253008 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:11:32.254099 29895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:11:32.373250 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 29697
I20260809 13:11:32.397413 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 29072
I20260809 13:11:32.398689 29180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:11:32.544061 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 29072
I20260809 13:11:32.575640 10966 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 13:11:32.920089 10966 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 13:11:32.920339 10966 mini_ranger.cc:66] Stopping Ranger...
I20260809 13:11:33.264794 10966 mini_ranger.cc:68] Stopped Ranger
2026-08-09 13:11:33.265 UTC [28385] LOG: received smart shutdown request
2026-08-09 13:11:33.269 UTC [28385] LOG: background worker "logical replication launcher" (PID 28401) exited with exit code 1
2026-08-09 13:11:33.269 UTC [28392] LOG: shutting down
2026-08-09 13:11:33.400 UTC [28385] LOG: database system is shut down
2026-08-09T13:11:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48576 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:11:33 dist-test-slave-42z9 krb5kdc[29985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:11:33 dist-test-slave-42z9 krb5kdc[29985](info): set up 2 sockets
Aug 09 13:11:33 dist-test-slave-42z9 krb5kdc[29985](info): commencing operation
krb5kdc: starting...
W20260809 13:11:34.734320 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.008s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:11:34 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:11:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangeradmin_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangeradmin_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangerlookup_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangerlookup_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
I20260809 13:11:35.098434 10966 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPW_N6O/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/postgres -l logfile start
2026-08-09 13:11:37.266 UTC [30037] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 13:11:37.266 UTC [30037] LOG: listening on IPv4 address "127.10.181.148", port 35127
2026-08-09 13:11:37.269 UTC [30037] LOG: listening on Unix socket "/tmp/.s.PGSQL.35127"
2026-08-09 13:11:37.290 UTC [30039] LOG: database system was shut down at 2026-08-09 13:11:36 UTC
2026-08-09 13:11:37.293 UTC [30037] LOG: database system is ready to accept connections
I20260809 13:11:38.383188 10966 mini_postgres.cc:96] Postgres bound to 35127
2026-08-09 13:11:38.406 UTC [30057] FATAL: database "slave" does not exist
127.10.181.148:35127 - accepting connections
I20260809 13:11:38.407006 10966 mini_ranger.cc:161] Starting Ranger...
I20260809 13:11:38.435552 10966 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 13:11:38.830947 10966 mini_ranger.cc:87] Created ranger Postgres database
I20260809 13:11:38.831069 10966 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin
2026-08-09 13:11:39,128 [I] DB FLAVOR :POSTGRES
2026-08-09 13:11:39,128 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 13:11:39,128 [I] Checking connection..
2026-08-09 13:11:39,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 13:11:39,392 [I] Checking connection passed.
2026-08-09 13:11:39,393 [I] --------- Verifying version history table ---------
2026-08-09 13:11:39,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:11:39,632 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 13:11:39,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:11:39,877 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 13:11:39,877 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 13:11:39,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 13:11:40.130 UTC [30158] WARNING: there is no transaction in progress
2026-08-09 13:11:40,136 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 13:11:40,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 13:11:40,393 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 13:11:40,393 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 13:11:40,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 13:11:40,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 13:11:40,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-42z9.c.gcp-upstream.internal','N') ;"
2026-08-09 13:11:41,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 13:11:41,375 [I] Table x_portal_user does not exist in database ranger
2026-08-09 13:11:41,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 13:11:41,612 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 13:11:41,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 13:11:41,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 13:11:42,117 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 13:11:42,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 13:11:42.984 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:42.998 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.017 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.030 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.046 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.064 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.115 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.120 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.133 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.142 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.151 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.164 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.172 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.178 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.183 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43.188 UTC [30321] WARNING: there is no transaction in progress
2026-08-09 13:11:43,724 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 13:11:43,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 13:11:43,951 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 13:11:43,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 13:11:44,208 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 13:11:44,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 13:11:44,464 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 13:11:44,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 13:11:44,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 13:11:44,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-42z9.c.gcp-upstream.internal';"
2026-08-09 13:11:45,156 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 13:11:45,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 13:11:45,407 [I] DB_PATCHES have already been applied
I20260809 13:11:45.412818 10966 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPW_N6O/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 13:11:45.412906 10966 mini_ranger.cc:193] Using host: 127.10.181.148
I20260809 13:11:45.433678 10966 mini_ranger.cc:227] Ranger admin URL: http://127.10.181.148:39805
Aug 09, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 1:11:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 1:11:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 1:11:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 13:11:46.709623 10966 mini_ranger.cc:160] Time spent starting Ranger: real 8.303s user 0.000s sys 0.060s
Aug 09, 2026 1:12:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 1:12:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 1:12:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 1:12:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 1:12:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 13:12:07.714031 10966 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.181.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.181.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangerkms_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.181.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/rangerkms_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.181.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.181.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.181.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/HTTP_127.10.181.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 13:12:07 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281127, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 13:12:07.935429 10966 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 13:12:07.961418 10966 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 13:12:08.732759 10966 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 13:12:08.732854 10966 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms
I20260809 13:12:08.732885 10966 mini_ranger_kms.cc:227] Using postgres at 127.10.181.148:35127
2026-08-09 13:12:09,089 [I] DB FLAVOR :POSTGRES
2026-08-09 13:12:09,089 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 13:12:09,089 [I] Checking connection
2026-08-09 13:12:09,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 13:12:09,321 [I] connection success
2026-08-09 13:12:09,321 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 13:12:09,321 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 13:12:09,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 13:12:09,553 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 13:12:09,553 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 13:12:09,554 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 13:12:09,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.181.148:35127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 13:12:09,805 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 13:12:10.270102 10966 mini_ranger_kms.cc:310] Created kms service
I20260809 13:12:10.435664 10966 mini_ranger_kms.cc:326] Created kudu user
I20260809 13:12:10.481459 10966 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 13:12:10 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281127, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:10.869989 10966 mini_ranger_kms.cc:384] Added ranger policy
I20260809 13:12:10.870373 10966 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPW_N6O/build/asan/bin/postgresql.jar:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPW_N6O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 13:12:10.870434 10966 mini_ranger_kms.cc:240] Using host: 127.10.181.148
I20260809 13:12:10.892014 10966 mini_ranger_kms.cc:276] Ranger KMS PID: 30610
I20260809 13:12:10.892158 10966 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.181.148:47203
Aug 09, 2026 1:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 1:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 1:12:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 1:12:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 13:12:13 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786281133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:13.385459 10966 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.450s user 0.076s sys 0.318s
I20260809 13:12:13.385722 10966 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 13:12:13.385913 10966 mini_ranger_kms.cc:401] 127.10.181.148:47203/kms/v1/keys
Aug 09 13:12:13 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
Aug 09 13:12:14 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.181.148@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
Aug 09 13:12:16 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:12:16.408550 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39615
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39615
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:16.745066 30689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:16.745369 30689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:16.745534 30689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:16.753147 30689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:16.753271 30689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:16.753348 30689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:16.753422 30689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:16.753522 30689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:16.763785 30689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.181.148:47203/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39615
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:16.765101 30689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:16.766489 30689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:16.779917 30696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:16.779896 30697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:16.780303 30699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:16.780962 30689 server_base.cc:1034] running on GCE node
I20260809 13:12:16.781934 30689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:16.783759 30689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:16.785013 30689 hybrid_clock.cc:648] HybridClock initialized: now 1786281136784960 us; error 41 us; skew 500 ppm
I20260809 13:12:16.785418 30689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:16 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:16.794000 30689 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:16.795861 30689 webserver.cc:460] Webserver started at http://127.10.181.190:41365/ using document root <none> and password file <none>
I20260809 13:12:16.796696 30689 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:16.796828 30689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:16.797191 30689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:12:16 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:16.850278 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "f30f43d6c18b42708623e36b52958d6b"
format_stamp: "Formatted at 2026-08-09 13:12:16 on dist-test-slave-42z9"
server_key: "7cfe812950f3f9faf541be3a9493d0e0"
server_key_iv: "c762e710dc1bf7200484ba234045d180"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:16.851397 30689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "f30f43d6c18b42708623e36b52958d6b"
format_stamp: "Formatted at 2026-08-09 13:12:16 on dist-test-slave-42z9"
server_key: "7cfe812950f3f9faf541be3a9493d0e0"
server_key_iv: "c762e710dc1bf7200484ba234045d180"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:16.858362 30689 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 13:12:16.877671 30711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:16.879349 30689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 13:12:16.879567 30689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "f30f43d6c18b42708623e36b52958d6b"
format_stamp: "Formatted at 2026-08-09 13:12:16 on dist-test-slave-42z9"
server_key: "7cfe812950f3f9faf541be3a9493d0e0"
server_key_iv: "c762e710dc1bf7200484ba234045d180"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:16.879812 30689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:16.912912 30689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:16.916901 30689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:16.917202 30689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:16.938051 30689 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39615
I20260809 13:12:16.938056 30798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39615 every 8 connection(s)
I20260809 13:12:16.939508 30689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:16.944278 30799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:16.949548 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 30689
I20260809 13:12:16.949765 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:12:16.950402 10966 external_mini_cluster.cc:1428] Setting key 56d4ab037ad9d3d0df6b9410beb9faca
I20260809 13:12:16.958439 30799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Bootstrap starting.
I20260809 13:12:16.962389 30799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:16.963824 30799 log.cc:824] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:16.967173 30799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: No bootstrap required, opened a new log
Aug 09 13:12:16 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:16.977969 30799 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:16.978408 30799 raft_consensus.cc:374] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:16.978521 30799 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f43d6c18b42708623e36b52958d6b, State: Initialized, Role: FOLLOWER
I20260809 13:12:16.979206 30799 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:16.979655 30799 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:16.979876 30799 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:16.980134 30799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:16.981561 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:16.952517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.10.181.190:39615)
0809 13:12:16.962074 (+ 9557us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:16.962090 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:16.962452 (+ 362us) server_negotiation.cc:407] Connection header received
0809 13:12:16.964418 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:16.964454 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:16.964966 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:16.965390 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:16.966770 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:16.968129 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:16.969360 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:16.970088 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:16.970282 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:16.974238 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:16.974247 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:16.974262 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:16.974297 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:16.977160 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:16.977861 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:16.977868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:16.977875 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:16.978026 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:16.978425 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:16.978430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:16.978433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:16.979129 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:16.979444 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:16.979911 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 13:12:16.980502 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9291,"thread_start_us":9020,"threads_started":1}
I20260809 13:12:16.983428 30799 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:16.984053 30799 leader_election.cc:302] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30f43d6c18b42708623e36b52958d6b; no voters:
I20260809 13:12:16.984786 30799 leader_election.cc:288] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:12:16.984948 30805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:16.986969 30799 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:16.987985 30805 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 LEADER]: Becoming Leader. State: Replica: f30f43d6c18b42708623e36b52958d6b, State: Running, Role: LEADER
I20260809 13:12:16.988550 30805 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:16.993995 30809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30f43d6c18b42708623e36b52958d6b. Latest consensus state: current_term: 1 leader_uuid: "f30f43d6c18b42708623e36b52958d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } } }
I20260809 13:12:16.994403 30809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:16.995563 30808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f30f43d6c18b42708623e36b52958d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } } }
I20260809 13:12:16.996417 30808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:17.010725 30814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:17.017675 30814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:17.038702 30814 catalog_manager.cc:1347] Generated new cluster ID: b959595748754d3c86b97c59b407b62c
I20260809 13:12:17.038830 30814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:17.081046 30814 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:12:17.083058 30814 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:17.094242 30814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Generated new TSK 0
I20260809 13:12:17.095410 30814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:12:17.097894 30799 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:12:17.845182 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 13:12:18.302065 30870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:18.302431 30870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:18.302613 30870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:18.313553 30870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:18.313680 30870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:18.313910 30870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:18.328703 30870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:18.330510 30870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:18.332309 30870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:18.350512 30885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:18.351059 30887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:18.359454 30870 server_base.cc:1034] running on GCE node
W20260809 13:12:18.359889 30892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:18.361320 30870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:18.362800 30870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:18.364115 30870 hybrid_clock.cc:648] HybridClock initialized: now 1786281138364043 us; error 59 us; skew 500 ppm
I20260809 13:12:18.364508 30870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:18.368607 30870 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:18.370561 30870 webserver.cc:460] Webserver started at http://127.10.181.129:44239/ using document root <none> and password file <none>
I20260809 13:12:18.371464 30870 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:18.371644 30870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:18.372032 30870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:18.391701 30870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "58004148e80c4a1ebb4ff41230d3daba"
format_stamp: "Formatted at 2026-08-09 13:12:18 on dist-test-slave-42z9"
server_key: "de05b31be9bd0fe46119c2976d4bf9ea"
server_key_iv: "d98e76dcfd17959df735bd8a204fd057"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:18.392814 30870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "58004148e80c4a1ebb4ff41230d3daba"
format_stamp: "Formatted at 2026-08-09 13:12:18 on dist-test-slave-42z9"
server_key: "de05b31be9bd0fe46119c2976d4bf9ea"
server_key_iv: "d98e76dcfd17959df735bd8a204fd057"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:18.411244 30870 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.010s sys 0.007s
I20260809 13:12:18.443769 30901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:18.445705 30870 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.009s
I20260809 13:12:18.445909 30870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "58004148e80c4a1ebb4ff41230d3daba"
format_stamp: "Formatted at 2026-08-09 13:12:18 on dist-test-slave-42z9"
server_key: "de05b31be9bd0fe46119c2976d4bf9ea"
server_key_iv: "d98e76dcfd17959df735bd8a204fd057"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:18.446183 30870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:18.479715 30870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:18.483615 30870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:18.483891 30870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:18.484929 30870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:18.486521 30870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:18.486598 30870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:18.486709 30870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:18.486766 30870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:18.563922 30870 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:33233
I20260809 13:12:18.563966 31077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:33233 every 8 connection(s)
I20260809 13:12:18.565754 30870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:18.573261 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 30870
I20260809 13:12:18.573482 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:12:18.574216 10966 external_mini_cluster.cc:1428] Setting key f42f9931c39725ce4b33e8bd4761d3c0
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:18.657931 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:18.572690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:47393 (local address 127.10.181.190:39615)
0809 13:12:18.617732 (+ 45042us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:18.617740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:18.620838 (+ 3098us) server_negotiation.cc:407] Connection header received
0809 13:12:18.632495 (+ 11657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:18.632502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:18.632621 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:18.632710 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:18.635335 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:18.636514 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:18.639705 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:18.640365 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:18.640486 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:18.649705 (+ 9219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:18.649713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:18.649718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:18.649776 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:18.653858 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:18.654813 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:18.654818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:18.654822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:18.655012 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:18.655589 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:18.655594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:18.655597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:18.655944 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:18.656098 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:18.657248 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0809 13:12:18.657520 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44876,"thread_start_us":42516,"threads_started":1}
I20260809 13:12:18.666914 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:18.572041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.129:47393)
0809 13:12:18.618482 (+ 46441us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:18.618550 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:18.632102 (+ 13552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:18.633335 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:18.633348 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:18.634176 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:18.635118 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:18.635146 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:18.636723 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:18.636728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:18.639514 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:18.639531 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:18.644952 (+ 5421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:18.644960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:18.645264 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:18.646308 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:18.646342 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:18.649376 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:18.654050 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:18.654060 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:18.654079 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:18.654661 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:18.655191 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:18.655196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:18.655200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:18.655450 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:18.656202 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:18.656210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:18.656744 (+ 534us) client_negotiation.cc:769] Sending connection context
0809 13:12:18.664868 (+ 8124us) client_negotiation.cc:240] Negotiation successful
0809 13:12:18.665513 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46023,"thread_start_us":55849,"threads_started":1}
I20260809 13:12:18.668920 31078 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:18.669256 31078 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:18.670230 31078 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:18.672932 30737 ts_manager.cc:194] Registered new tserver with Master: 58004148e80c4a1ebb4ff41230d3daba (127.10.181.129:33233)
I20260809 13:12:18.676000 30737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:47393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
W20260809 13:12:18.698292 10966 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.181.130: real 0.115s user 0.000s sys 0.025s
I20260809 13:12:18.699175 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:18 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
W20260809 13:12:19.177564 31088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:19.178114 31088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:19.178331 31088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:19.190096 31088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:19.190318 31088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:19.190557 31088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:19.206444 31088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:19.208478 31088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:19.210261 31088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:19.229266 31100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:19.230319 31102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:19.238095 31088 server_base.cc:1034] running on GCE node
W20260809 13:12:19.238773 31105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:19.239560 31088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:19.240662 31088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:19.241885 31088 hybrid_clock.cc:648] HybridClock initialized: now 1786281139241810 us; error 63 us; skew 500 ppm
I20260809 13:12:19.242323 31088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:19.246840 31088 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:19.267210 31088 webserver.cc:460] Webserver started at http://127.10.181.130:36475/ using document root <none> and password file <none>
I20260809 13:12:19.268289 31088 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:19.268458 31088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:19.268855 31088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:19.286958 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "0e5c20fa6d0e4bd39d5120a668756099"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "146f57262986b26f50335d88781dddcb"
server_key_iv: "273a6d1c1db598f4e34b2865d217018b"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.288053 31088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "0e5c20fa6d0e4bd39d5120a668756099"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "146f57262986b26f50335d88781dddcb"
server_key_iv: "273a6d1c1db598f4e34b2865d217018b"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.293649 31088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 13:12:19.309984 31128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.311645 31088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:12:19.311826 31088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "0e5c20fa6d0e4bd39d5120a668756099"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "146f57262986b26f50335d88781dddcb"
server_key_iv: "273a6d1c1db598f4e34b2865d217018b"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.312070 31088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:19.341915 31088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:19.347172 31088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:19.347524 31088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:19.348757 31088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:19.350683 31088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:19.350785 31088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.350912 31088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:19.350989 31088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.388294 31088 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:36685
I20260809 13:12:19.388324 31293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:36685 every 8 connection(s)
I20260809 13:12:19.390031 31088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:19.400326 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 31088
I20260809 13:12:19.400503 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:12:19.401103 10966 external_mini_cluster.cc:1428] Setting key 3e457d0c03ac98457a1977a25237f7e1
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:19.412014 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:19.392645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:47117 (local address 127.10.181.190:39615)
0809 13:12:19.393180 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:19.393187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:19.394153 (+ 966us) server_negotiation.cc:407] Connection header received
0809 13:12:19.395984 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:19.395990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:19.396090 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:19.396227 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:19.397553 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.398776 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.400330 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.400833 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.400956 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:19.404980 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:19.404986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:19.404990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:19.405029 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:19.408325 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.409231 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.409237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.409240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.409381 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.410035 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.410041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.410044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.410394 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:19.410543 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:19.411413 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 13:12:19.411673 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":259,"threads_started":1}
I20260809 13:12:19.413225 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:19.393169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.130:47117)
0809 13:12:19.393973 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:19.394026 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:19.395717 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:19.396379 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:19.396387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:19.396823 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:19.397367 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:19.397383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.398940 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.398944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:19.400192 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:19.400205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.401071 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.401079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:19.401332 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:19.402027 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:19.402057 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:19.404737 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:19.408480 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:19.408489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:19.408507 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:19.409089 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:19.409527 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:19.409532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:19.409535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:19.409813 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:19.410546 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:19.410554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:19.410975 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 13:12:19.411298 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 13:12:19.411812 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":337,"threads_started":1}
I20260809 13:12:19.414695 31294 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:19.415061 31294 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:19.415911 31294 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:19.417708 30737 ts_manager.cc:194] Registered new tserver with Master: 0e5c20fa6d0e4bd39d5120a668756099 (127.10.181.130:36685)
I20260809 13:12:19.418743 30737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:47117
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:12:19.462340 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:19.679620 31078 heartbeater.cc:503] Master 127.10.181.190:39615 was elected leader, sending a full tablet report...
W20260809 13:12:19.796527 31301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:19.796842 31301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:19.796979 31301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:19.803944 31301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:19.804051 31301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:19.804193 31301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:12:19.813606 31301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:19.814836 31301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:19.816067 31301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:19.828554 31310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:19.828651 31309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:19.828609 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:19.829239 31301 server_base.cc:1034] running on GCE node
I20260809 13:12:19.829823 31301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:19.830667 31301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:19.831854 31301 hybrid_clock.cc:648] HybridClock initialized: now 1786281139831793 us; error 53 us; skew 500 ppm
I20260809 13:12:19.832149 31301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:19.835091 31301 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:12:19.836481 31301 webserver.cc:460] Webserver started at http://127.10.181.131:41365/ using document root <none> and password file <none>
I20260809 13:12:19.837210 31301 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:19.837328 31301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:19.837687 31301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:19.852830 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "52e1c24efdc142789509f4c98ff87362"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "ccd49918362102d13a1ff099ac8c84b7"
server_key_iv: "bcfe804856f286d33c9dc9110dd1af2d"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.853919 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "52e1c24efdc142789509f4c98ff87362"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "ccd49918362102d13a1ff099ac8c84b7"
server_key_iv: "bcfe804856f286d33c9dc9110dd1af2d"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.860131 31301 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 13:12:19.877189 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.878818 31301 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 13:12:19.878989 31301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "52e1c24efdc142789509f4c98ff87362"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "ccd49918362102d13a1ff099ac8c84b7"
server_key_iv: "bcfe804856f286d33c9dc9110dd1af2d"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:19.879210 31301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:19.893872 31301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:19.897820 31301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:19.898098 31301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:19.899049 31301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:19.900434 31301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:19.900508 31301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.900631 31301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:19.900686 31301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:19.930794 31301 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:34977
I20260809 13:12:19.930825 31493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:34977 every 8 connection(s)
I20260809 13:12:19.932116 31301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:12:19.938984 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 31301
I20260809 13:12:19.939158 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:12:19.939649 10966 external_mini_cluster.cc:1428] Setting key e6feb3321c0b28fb1035dab386a6ae9d
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281139, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:19.952844 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:19.934642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:51353 (local address 127.10.181.190:39615)
0809 13:12:19.935081 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:19.935086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:19.935996 (+ 910us) server_negotiation.cc:407] Connection header received
0809 13:12:19.937820 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:19.937824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:19.937913 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:19.938013 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:19.939929 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.940840 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.943112 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.943565 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.943658 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:19.947305 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:19.947309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:19.947313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:19.947345 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:19.949872 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.950625 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.950629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.950631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.950756 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.951188 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.951191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.951193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.951490 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:19.951628 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:19.952274 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 13:12:19.952485 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":206,"threads_started":1}
I20260809 13:12:19.953859 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:19.935142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.131:51353)
0809 13:12:19.935833 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:19.935871 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:19.937541 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:19.938299 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:19.938310 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:19.938951 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:19.939702 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:19.939725 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.941002 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.941008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:19.942972 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:19.942987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.943783 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.943791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:19.944064 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:19.944728 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:19.944757 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:19.947097 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:19.949999 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:19.950008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:19.950023 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:19.950506 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:19.950860 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:19.950865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:19.950869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:19.951073 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:19.951608 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:19.951615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:19.951917 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 13:12:19.952144 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 13:12:19.952602 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":268,"threads_started":1}
I20260809 13:12:19.955128 31495 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:19.955382 31495 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:19.956027 31495 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:19.957357 30736 ts_manager.cc:194] Registered new tserver with Master: 52e1c24efdc142789509f4c98ff87362 (127.10.181.131:34977)
I20260809 13:12:19.958264 30736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:51353
I20260809 13:12:19.966809 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:12:19.968379 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 30870
Aug 09 13:12:19 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:12:19.983964 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:19.969152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.10.181.129:33233)
0809 13:12:19.969824 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:19.969830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:19.969853 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:12:19.969909 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:19.969915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:19.970078 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:19.970211 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:19.971628 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.972661 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.974211 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:19.974733 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:19.974845 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:19.978884 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:19.978895 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:19.978907 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:19.978938 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:19.981332 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.981993 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.982001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.982003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.982143 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:19.982520 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:19.982524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:19.982526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:19.982888 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:19.983165 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:19.983454 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 13:12:19.983660 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":305,"threads_started":1}
I20260809 13:12:19.985074 31071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:20.065567 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 30870
I20260809 13:12:20.089287 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 31088
Aug 09 13:12:20 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:12:20.110615 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:20.090164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.10.181.130:36685)
0809 13:12:20.090669 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:20.090676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:20.090704 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:12:20.090768 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:20.090774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:20.090988 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:20.091131 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:20.092331 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:20.093297 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:20.094710 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:20.095272 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:20.095363 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:20.103993 (+ 8630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:20.104010 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:20.104028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:20.104083 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:20.107598 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:20.108326 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:20.108334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:20.108336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:20.108491 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:20.108854 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:20.108860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:20.108864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:20.109238 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:20.109540 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:20.109982 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 13:12:20.110247 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260809 13:12:20.112156 31282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:20.193066 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 31088
I20260809 13:12:20.214831 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 31301
Aug 09 13:12:20 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:12:20.230716 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:20.215760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.10.181.131:34977)
0809 13:12:20.216406 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:20.216414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:20.216445 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:12:20.216513 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:20.216520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:20.216733 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:20.216880 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:20.218142 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:20.219184 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:20.220407 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:20.220943 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:20.221040 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:20.225082 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:20.225094 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:20.225109 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:20.225150 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:20.227846 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:20.228480 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:20.228488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:20.228490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:20.228624 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:20.229048 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:20.229051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:20.229058 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:20.229432 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:20.229793 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:20.230133 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 13:12:20.230389 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":292,"threads_started":1}
I20260809 13:12:20.231942 31485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:20.313500 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 31301
I20260809 13:12:20.336390 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 30689
I20260809 13:12:20.337683 30792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:20.446806 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 30689
I20260809 13:12:20.473697 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39615
--webserver_interface=127.10.181.190
--webserver_port=41365
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39615
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:20.800663 31512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:20.800949 31512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:20.801090 31512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:20.808130 31512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:20.808239 31512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:20.808316 31512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:20.808393 31512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:20.808457 31512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:20.817996 31512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39615
--ranger_config_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:20.819233 31512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:20.820530 31512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:20.831403 31523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:20.831534 31521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:20.831497 31520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:20.832471 31512 server_base.cc:1034] running on GCE node
I20260809 13:12:20.833177 31512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:20.834487 31512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:20.835685 31512 hybrid_clock.cc:648] HybridClock initialized: now 1786281140835643 us; error 28 us; skew 500 ppm
I20260809 13:12:20.836005 31512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:20 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281140, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:20.838912 31512 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:20.840250 31512 webserver.cc:460] Webserver started at http://127.10.181.190:41365/ using document root <none> and password file <none>
I20260809 13:12:20.840868 31512 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:20.840988 31512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 13:12:20 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281140, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:20.881916 31512 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 13:12:20.884817 31535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:20.886143 31512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:12:20.886318 31512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "f30f43d6c18b42708623e36b52958d6b"
format_stamp: "Formatted at 2026-08-09 13:12:16 on dist-test-slave-42z9"
server_key: "7cfe812950f3f9faf541be3a9493d0e0"
server_key_iv: "c762e710dc1bf7200484ba234045d180"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:20.887261 31512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:20.898512 31512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:20.902109 31512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:20.902376 31512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:20.920115 31512 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39615
I20260809 13:12:20.920138 31608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39615 every 8 connection(s)
I20260809 13:12:20.921577 31512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:20.928319 31610 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:12:20.928696 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 31512
I20260809 13:12:20.929774 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:33233
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:39615
--webserver_port=44239
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:20.932736 31610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Bootstrap starting.
I20260809 13:12:20.953662 31610 log.cc:824] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:20.962954 31610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:12:20.963445 31610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Bootstrap complete.
I20260809 13:12:20.972362 31610 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:20.973170 31610 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f43d6c18b42708623e36b52958d6b, State: Initialized, Role: FOLLOWER
I20260809 13:12:20.973819 31610 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:20.974023 31610 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:20.974138 31610 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:20.974289 31610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:12:20.977941 31610 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:20.978385 31610 leader_election.cc:302] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30f43d6c18b42708623e36b52958d6b; no voters:
I20260809 13:12:20.978948 31610 leader_election.cc:288] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 13:12:20.979108 31618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:12:20.980238 31618 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [term 2 LEADER]: Becoming Leader. State: Replica: f30f43d6c18b42708623e36b52958d6b, State: Running, Role: LEADER
I20260809 13:12:20.980811 31618 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } }
I20260809 13:12:20.981096 31610 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:20.989784 31619 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f30f43d6c18b42708623e36b52958d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } } }
I20260809 13:12:20.991654 31619 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:20.990221 31621 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30f43d6c18b42708623e36b52958d6b. Latest consensus state: current_term: 2 leader_uuid: "f30f43d6c18b42708623e36b52958d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f43d6c18b42708623e36b52958d6b" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39615 } } }
I20260809 13:12:20.992274 31621 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:21.002470 31638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:21.002950 31610 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 13:12:21.016655 31638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:21.018723 31638 catalog_manager.cc:1259] Loaded cluster ID: b959595748754d3c86b97c59b407b62c
I20260809 13:12:21.018886 31638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:21.023705 31638 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:21.025416 31638 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Loaded TSK: 0
I20260809 13:12:21.027163 31638 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 13:12:21.378435 31612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:21.378963 31612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:21.379202 31612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:21.390465 31612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:21.390713 31612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:21.391001 31612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:21.406441 31612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:33233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=44239
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:21.408488 31612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:21.410328 31612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:21.432224 31672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:21.434015 31674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:21.439992 31612 server_base.cc:1034] running on GCE node
W20260809 13:12:21.440951 31678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:21.441887 31612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:21.443163 31612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:21.444434 31612 hybrid_clock.cc:648] HybridClock initialized: now 1786281141444367 us; error 61 us; skew 500 ppm
I20260809 13:12:21.444779 31612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:21 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281141, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:21.448007 31612 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:21.449939 31612 webserver.cc:460] Webserver started at http://127.10.181.129:44239/ using document root <none> and password file <none>
I20260809 13:12:21.450706 31612 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:21.450847 31612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 13:12:21 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281141, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:21.495008 31612 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.005s
I20260809 13:12:21.505523 31690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:21.507328 31612 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.006s
I20260809 13:12:21.507572 31612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "58004148e80c4a1ebb4ff41230d3daba"
format_stamp: "Formatted at 2026-08-09 13:12:18 on dist-test-slave-42z9"
server_key: "de05b31be9bd0fe46119c2976d4bf9ea"
server_key_iv: "d98e76dcfd17959df735bd8a204fd057"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:21.509052 31612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:21.548925 31612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:21.554790 31612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:21.555222 31612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:21.564622 31612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:21.573072 31612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:21.573294 31612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 13:12:21.573539 31612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:21.573695 31612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:21.629504 31612 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:33233
I20260809 13:12:21.629537 31866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:33233 every 8 connection(s)
I20260809 13:12:21.631553 31612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:21.635998 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 31612
I20260809 13:12:21.636830 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:36685
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:39615
--webserver_port=36475
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:21 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281141, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:21.675978 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:21.634934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.129:58173)
0809 13:12:21.648145 (+ 13211us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:21.648211 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:21.651965 (+ 3754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:21.654640 (+ 2675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:21.654651 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:21.655330 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:21.656089 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:21.656115 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:21.658396 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:21.658401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:21.660320 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:21.660338 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:21.662110 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:21.662130 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:21.662391 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:21.663226 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:21.663263 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:21.665967 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:21.670473 (+ 4506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:21.670484 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:21.670502 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:21.671108 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:21.671698 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:21.671703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:21.671706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:21.671995 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:21.673185 (+ 1190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:21.673194 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:21.673663 (+ 469us) client_negotiation.cc:769] Sending connection context
0809 13:12:21.674000 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 13:12:21.674608 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12891,"thread_start_us":12653,"threads_started":1}
I20260809 13:12:21.677917 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:21.635806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:58173 (local address 127.10.181.190:39615)
0809 13:12:21.647496 (+ 11690us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:21.647521 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:21.651426 (+ 3905us) server_negotiation.cc:407] Connection header received
0809 13:12:21.652914 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:21.652934 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:21.653487 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:21.653955 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:21.656342 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:21.658192 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:21.660519 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:21.661499 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:21.661665 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:21.666302 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:21.666313 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:21.666333 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:21.666383 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:21.670284 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:21.671314 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:21.671325 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:21.671334 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:21.671544 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:21.672162 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:21.672168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:21.672171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:21.672869 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:21.673072 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:21.676156 (+ 3084us) server_negotiation.cc:299] Negotiation successful
0809 13:12:21.676607 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11412,"thread_start_us":11185,"threads_started":1}
I20260809 13:12:21.681006 31867 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:21.681344 31867 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:21.682262 31867 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
W20260809 13:12:21.685415 31867 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:39615 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 13:12:21.685663 31867 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:21.685927 31867 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:21.687173 31867 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:21.687501 31867 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
W20260809 13:12:21.688650 31867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 13:12:22.052425 31873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:22.052724 31873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:22.052891 31873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:22.060282 31873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:22.060411 31873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:22.060597 31873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:22.070837 31873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:36685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=36475
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:22.072156 31873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:22.073557 31873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:22.089418 31894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:22.097803 31896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:22.099532 31898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:22.100273 31873 server_base.cc:1034] running on GCE node
I20260809 13:12:22.101249 31873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:22.113721 31873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:22.115091 31873 hybrid_clock.cc:648] HybridClock initialized: now 1786281142115008 us; error 73 us; skew 500 ppm
I20260809 13:12:22.115587 31873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:22.124631 31873 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:22.137671 31873 webserver.cc:460] Webserver started at http://127.10.181.130:36475/ using document root <none> and password file <none>
I20260809 13:12:22.139056 31873 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:22.139366 31873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:22.173720 31873 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:12:22.177357 31914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.179157 31873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 13:12:22.179348 31873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "0e5c20fa6d0e4bd39d5120a668756099"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "146f57262986b26f50335d88781dddcb"
server_key_iv: "273a6d1c1db598f4e34b2865d217018b"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:22.180773 31873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:22.207518 31873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:22.212661 31873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:22.213032 31873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:22.216321 31873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:22.221702 31873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:22.221863 31873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.222018 31873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:22.222138 31873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.286405 31873 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:36685
I20260809 13:12:22.286456 32095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:36685 every 8 connection(s)
I20260809 13:12:22.288072 31873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:22.297835 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 31873
I20260809 13:12:22.298708 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.181.148:47203/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:34977
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:39615
--webserver_port=41365
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:22.317041 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.292237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:50713 (local address 127.10.181.190:39615)
0809 13:12:22.297670 (+ 5433us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:22.297677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:22.297699 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:12:22.297765 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:22.297771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:22.297858 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:22.297967 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:22.301838 (+ 3871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.302839 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.305017 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.305617 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.305717 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.309864 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:22.309870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:22.309874 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:22.309912 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:22.313191 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.314137 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.314143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.314146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.314312 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.314940 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.314945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.314948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.315278 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:22.315409 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:22.316404 (+ 995us) server_negotiation.cc:299] Negotiation successful
0809 13:12:22.316637 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5307,"thread_start_us":10687,"threads_started":1}
I20260809 13:12:22.319341 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.291663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.130:50713)
0809 13:12:22.294020 (+ 2357us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:22.294076 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:22.296587 (+ 2511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:22.300214 (+ 3627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:22.300225 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:22.300815 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:22.301633 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:22.301658 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.303063 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.303068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:22.304843 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:22.304870 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.306046 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.306052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:22.306302 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.307056 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:22.307086 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:22.309615 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:22.313352 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:22.313360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:22.313379 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:22.313995 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:22.314488 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:22.314492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:22.314495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:22.314806 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:22.315504 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:22.315512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:22.315940 (+ 428us) client_negotiation.cc:769] Sending connection context
0809 13:12:22.317356 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0809 13:12:22.317948 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2068,"thread_start_us":1826,"threads_started":1}
I20260809 13:12:22.321112 32096 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:22.321431 32096 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:22.322182 32096 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:22.324290 31559 ts_manager.cc:194] Registered new tserver with Master: 0e5c20fa6d0e4bd39d5120a668756099 (127.10.181.130:36685)
I20260809 13:12:22.326354 31559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:50713
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:22.688787 31867 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:22.689239 31867 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:22.690605 31559 ts_manager.cc:194] Registered new tserver with Master: 58004148e80c4a1ebb4ff41230d3daba (127.10.181.129:33233)
I20260809 13:12:22.691783 31559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:58173
W20260809 13:12:22.725481 32102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:22.725980 32102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:22.726202 32102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:22.733911 32102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:22.734041 32102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:22.734196 32102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:12:22.744208 32102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.181.148:47203/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:34977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=41365
--tserver_master_addrs=127.10.181.190:39615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:22.745566 32102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:22.746959 32102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:22.761502 32108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:22.767443 32102 server_base.cc:1034] running on GCE node
W20260809 13:12:22.767628 32109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:22.768103 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:22.768921 32102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:22.770251 32102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:22.771520 32102 hybrid_clock.cc:648] HybridClock initialized: now 1786281142771447 us; error 56 us; skew 500 ppm
I20260809 13:12:22.771889 32102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:22.775633 32102 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:12:22.777420 32102 webserver.cc:460] Webserver started at http://127.10.181.131:41365/ using document root <none> and password file <none>
I20260809 13:12:22.778200 32102 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:22.778329 32102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
I20260809 13:12:22.800330 32102 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:12:22.804030 32127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.805785 32102 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 13:12:22.805979 32102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "52e1c24efdc142789509f4c98ff87362"
format_stamp: "Formatted at 2026-08-09 13:12:19 on dist-test-slave-42z9"
server_key: "ccd49918362102d13a1ff099ac8c84b7"
server_key_iv: "bcfe804856f286d33c9dc9110dd1af2d"
server_key_version: "kuduclusterkey@0"
I20260809 13:12:22.807360 32102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:22.832348 32102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:22.837569 32102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:22.837884 32102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:22.839732 32102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:22.841658 32102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:22.841771 32102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.841887 32102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:22.841974 32102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:22.891358 32102 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:34977
I20260809 13:12:22.891440 32289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:34977 every 8 connection(s)
I20260809 13:12:22.893505 32102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:12:22.898422 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 32102
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:22.926358 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.897513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:43387 (local address 127.10.181.190:39615)
0809 13:12:22.899556 (+ 2043us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:22.899563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:22.906098 (+ 6535us) server_negotiation.cc:407] Connection header received
0809 13:12:22.907878 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:22.907884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:22.907981 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:22.908105 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:22.910275 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.911392 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.913776 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.914418 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.914514 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.919447 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:22.919453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:22.919458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:22.919495 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:22.922835 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.923690 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.923696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.923700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.923833 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.924313 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.924318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.924321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.924672 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:22.924801 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:22.925724 (+ 923us) server_negotiation.cc:299] Negotiation successful
0809 13:12:22.925966 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1899,"thread_start_us":1723,"threads_started":1}
I20260809 13:12:22.928494 32296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.896898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39615 (local address 127.10.181.131:43387)
0809 13:12:22.905925 (+ 9027us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:22.905974 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:22.907605 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:22.908598 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:22.908611 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:22.909199 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:22.910058 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:22.910085 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.911598 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.911604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:22.913604 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:22.913621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.914924 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.914932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:22.915223 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.915998 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:22.916029 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:22.919166 (+ 3137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:22.922986 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:22.922995 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:22.923013 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:22.923547 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:22.923944 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:22.923949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:22.923953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:22.924172 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:22.924875 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:22.924883 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:22.925273 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 13:12:22.926647 (+ 1374us) client_negotiation.cc:240] Negotiation successful
0809 13:12:22.927183 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8766,"thread_start_us":8574,"threads_started":1}
I20260809 13:12:22.930286 32291 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39615
I20260809 13:12:22.930624 32291 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:22.931546 32291 heartbeater.cc:511] Master 127.10.181.190:39615 requested a full tablet report, sending...
I20260809 13:12:22.933683 31559 ts_manager.cc:194] Registered new tserver with Master: 52e1c24efdc142789509f4c98ff87362 (127.10.181.131:34977)
I20260809 13:12:22.935135 31559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:43387
I20260809 13:12:22.944589 32299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.899445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.10.181.190:39615)
0809 13:12:22.908303 (+ 8858us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:22.908309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:22.912309 (+ 4000us) server_negotiation.cc:407] Connection header received
0809 13:12:22.912366 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:22.912371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:22.912450 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:22.912544 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:22.914911 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.915960 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.917775 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.918238 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.918345 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.932605 (+ 14260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:22.932613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:22.932618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:22.932658 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:22.938850 (+ 6192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.940536 (+ 1686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.940542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.940546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.940707 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.941513 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.941517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.941519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.941853 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:22.942115 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:22.944002 (+ 1887us) server_negotiation.cc:299] Negotiation successful
0809 13:12:22.944224 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8729,"thread_start_us":8580,"threads_started":1}
I20260809 13:12:22.947376 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:12:22.975430 32299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:22.952630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.10.181.190:39615)
0809 13:12:22.952818 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:22.952824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:22.958714 (+ 5890us) server_negotiation.cc:407] Connection header received
0809 13:12:22.959006 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:22.959013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:22.959094 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:22.959168 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:22.961348 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.962498 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.964516 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:22.965178 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:22.965273 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:22.968876 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:22.968883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:22.968888 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:22.968932 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:22.972362 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.973056 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.973062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.973065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.973220 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:22.973670 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:22.973676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:22.973679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:22.974049 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:22.974298 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:22.974870 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 13:12:22.975109 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:12:22.981252 31559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
Aug 09 13:12:22 dist-test-slave-42z9 krb5kdc[29985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281142, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for HTTP/127.10.181.148@KRBTEST.COM
W20260809 13:12:22.995291 31559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:12:23.078557 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.015915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:36685 (local address 127.0.0.1:47552)
0809 13:12:23.060454 (+ 44539us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:23.060513 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:23.060717 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:23.061884 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:23.061892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:23.061935 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:23.062354 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.062366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.065621 (+ 3255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.065627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.070172 (+ 4545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.070188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.077529 (+ 7341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.077538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.077757 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.077826 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 13:12:23.077966 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 13:12:23.078170 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44230,"thread_start_us":48966,"threads_started":1}
I20260809 13:12:23.079521 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.016554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.10.181.130:36685)
0809 13:12:23.059658 (+ 43104us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.059666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.060985 (+ 1319us) server_negotiation.cc:407] Connection header received
0809 13:12:23.061095 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.061103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.061379 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.061597 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:23.062581 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.064158 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.070367 (+ 6209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.071848 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.072010 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.072186 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.078965 (+ 6779us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.079163 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42928,"thread_start_us":43118,"threads_started":1}
I20260809 13:12:23.083341 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.017207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.10.181.129:33233)
0809 13:12:23.057511 (+ 40304us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.057521 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.064849 (+ 7328us) server_negotiation.cc:407] Connection header received
0809 13:12:23.066765 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.066775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.067030 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.067194 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:23.072377 (+ 5183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.074020 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.080037 (+ 6017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.081304 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.081509 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.081697 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.082855 (+ 1158us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.083035 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40050,"thread_start_us":40313,"threads_started":1}
I20260809 13:12:23.082711 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.016734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33233 (local address 127.0.0.1:54964)
0809 13:12:23.064713 (+ 47979us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:23.064734 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:23.066536 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:23.067513 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:23.067518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:23.067546 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:23.067877 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.067889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.074252 (+ 6363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.074258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.076295 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.076312 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.081909 (+ 5597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.081916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.082087 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.082125 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 13:12:23.082203 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 13:12:23.082339 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47839,"thread_start_us":47709,"threads_started":1}
I20260809 13:12:23.086279 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.065355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:34977 (local address 127.0.0.1:40818)
0809 13:12:23.077433 (+ 12078us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:23.077472 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:23.077614 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:23.078865 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:23.078870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:23.078897 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:23.079184 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.079196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.081293 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.081298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.083422 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.083439 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.085561 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.085568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.085731 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.085764 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:12:23.085851 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 13:12:23.085983 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11939,"thread_start_us":11769,"threads_started":1}
I20260809 13:12:23.087034 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.065774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.10.181.131:34977)
0809 13:12:23.076958 (+ 11184us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.076967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.077960 (+ 993us) server_negotiation.cc:407] Connection header received
0809 13:12:23.078085 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.078095 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.078371 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.078562 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:23.079446 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.081148 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.083609 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.085000 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.085191 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.085354 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.086527 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.086690 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10983,"thread_start_us":11917,"threads_started":1}
I20260809 13:12:23.088438 31767 tablet_service.cc:1467] Processing CreateTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 (DEFAULT_TABLE table=test-table [id=9f94dde72df9422390307ba242c48b7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:23.088291 31987 tablet_service.cc:1467] Processing CreateTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 (DEFAULT_TABLE table=test-table [id=9f94dde72df9422390307ba242c48b7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:23.089845 31987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b416ae5108e459ea6d7ee41ff1cca89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:23.089951 31767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b416ae5108e459ea6d7ee41ff1cca89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:23.093354 32190 tablet_service.cc:1467] Processing CreateTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 (DEFAULT_TABLE table=test-table [id=9f94dde72df9422390307ba242c48b7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:23.094763 32190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b416ae5108e459ea6d7ee41ff1cca89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:23.108613 32332 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Bootstrap starting.
I20260809 13:12:23.113519 32330 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Bootstrap starting.
I20260809 13:12:23.115763 32332 tablet_bootstrap.cc:654] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:23.117156 32332 log.cc:824] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:23.121273 32330 tablet_bootstrap.cc:654] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:23.122182 32333 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Bootstrap starting.
I20260809 13:12:23.123379 32330 log.cc:824] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:23.126942 32333 tablet_bootstrap.cc:654] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:23.128015 32330 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: No bootstrap required, opened a new log
I20260809 13:12:23.128244 32333 log.cc:824] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:23.128297 32332 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: No bootstrap required, opened a new log
I20260809 13:12:23.128331 32330 ts_tablet_manager.cc:1397] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260809 13:12:23.128702 32332 ts_tablet_manager.cc:1397] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.007s
I20260809 13:12:23.139571 32332 raft_consensus.cc:348] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.139705 32330 raft_consensus.cc:348] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.140192 32330 raft_consensus.cc:374] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:23.140324 32330 raft_consensus.cc:729] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58004148e80c4a1ebb4ff41230d3daba, State: Initialized, Role: FOLLOWER
I20260809 13:12:23.140949 32330 consensus_queue.cc:260] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.141335 32332 raft_consensus.cc:374] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:23.141523 32332 raft_consensus.cc:729] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5c20fa6d0e4bd39d5120a668756099, State: Initialized, Role: FOLLOWER
I20260809 13:12:23.142328 32332 consensus_queue.cc:260] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.143301 32333 tablet_bootstrap.cc:492] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: No bootstrap required, opened a new log
I20260809 13:12:23.143679 32333 ts_tablet_manager.cc:1397] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.012s
I20260809 13:12:23.152309 32332 ts_tablet_manager.cc:1428] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Time spent starting tablet: real 0.023s user 0.019s sys 0.000s
I20260809 13:12:23.152781 32096 heartbeater.cc:503] Master 127.10.181.190:39615 was elected leader, sending a full tablet report...
I20260809 13:12:23.158903 32333 raft_consensus.cc:348] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.159447 32333 raft_consensus.cc:374] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:23.159585 32333 raft_consensus.cc:729] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e1c24efdc142789509f4c98ff87362, State: Initialized, Role: FOLLOWER
I20260809 13:12:23.160388 32333 consensus_queue.cc:260] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.161830 32291 heartbeater.cc:503] Master 127.10.181.190:39615 was elected leader, sending a full tablet report...
I20260809 13:12:23.162631 32333 ts_tablet_manager.cc:1428] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Time spent starting tablet: real 0.019s user 0.009s sys 0.005s
I20260809 13:12:23.167830 31867 heartbeater.cc:503] Master 127.10.181.190:39615 was elected leader, sending a full tablet report...
I20260809 13:12:23.168560 32330 ts_tablet_manager.cc:1428] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Time spent starting tablet: real 0.040s user 0.022s sys 0.017s
I20260809 13:12:23.269884 32342 raft_consensus.cc:482] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:12:23.270279 32342 raft_consensus.cc:504] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.272565 32342 leader_election.cc:288] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52e1c24efdc142789509f4c98ff87362 (127.10.181.131:34977), 58004148e80c4a1ebb4ff41230d3daba (127.10.181.129:33233)
I20260809 13:12:23.291898 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.272933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:34977 (local address 127.10.181.130:53971)
0809 13:12:23.283739 (+ 10806us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:23.283762 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:23.284243 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:23.284855 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:23.284860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:23.284890 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:23.285309 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.285323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.286802 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.286807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.289108 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.289124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.291022 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.291028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.291236 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.291268 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:12:23.291352 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 13:12:23.291478 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10642,"mutex_wait_us":19,"thread_start_us":10938,"threads_started":1}
I20260809 13:12:23.292553 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.273097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:53971 (local address 127.10.181.131:34977)
0809 13:12:23.273428 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.273470 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.283867 (+ 10397us) server_negotiation.cc:407] Connection header received
0809 13:12:23.284450 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.284457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.284548 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.284619 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:23.285524 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.286629 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.289302 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.290578 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.290728 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.290803 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.292135 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.292266 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260809 13:12:23.294212 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b416ae5108e459ea6d7ee41ff1cca89" candidate_uuid: "0e5c20fa6d0e4bd39d5120a668756099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e1c24efdc142789509f4c98ff87362" is_pre_election: true
I20260809 13:12:23.294859 32230 raft_consensus.cc:2454] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e5c20fa6d0e4bd39d5120a668756099 in term 0.
I20260809 13:12:23.295647 31918 leader_election.cc:302] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5c20fa6d0e4bd39d5120a668756099, 52e1c24efdc142789509f4c98ff87362; no voters:
I20260809 13:12:23.296015 32342 raft_consensus.cc:2781] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:12:23.296152 32342 raft_consensus.cc:482] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:12:23.296237 32342 raft_consensus.cc:3037] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:23.299530 32342 raft_consensus.cc:504] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
W20260809 13:12:23.300541 32097 tablet.cc:2369] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:23.301635 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b416ae5108e459ea6d7ee41ff1cca89" candidate_uuid: "0e5c20fa6d0e4bd39d5120a668756099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52e1c24efdc142789509f4c98ff87362"
I20260809 13:12:23.301724 32342 leader_election.cc:288] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [CANDIDATE]: Term 1 election: Requested vote from peers 52e1c24efdc142789509f4c98ff87362 (127.10.181.131:34977), 58004148e80c4a1ebb4ff41230d3daba (127.10.181.129:33233)
I20260809 13:12:23.301839 32230 raft_consensus.cc:3037] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:23.305018 32230 raft_consensus.cc:2454] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e5c20fa6d0e4bd39d5120a668756099 in term 1.
I20260809 13:12:23.305557 31918 leader_election.cc:302] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5c20fa6d0e4bd39d5120a668756099, 52e1c24efdc142789509f4c98ff87362; no voters:
I20260809 13:12:23.305853 32342 raft_consensus.cc:2781] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:23.306481 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.273626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:33233 (local address 127.10.181.130:35479)
0809 13:12:23.290259 (+ 16633us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:23.290285 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:23.290497 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:23.291140 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:23.291145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:23.291170 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:23.298793 (+ 7623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.298807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.300287 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.300290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.303973 (+ 3683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:23.303990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.305628 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.305635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:23.305795 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.305831 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:12:23.305923 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 13:12:23.306055 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15683,"mutex_wait_us":3,"thread_start_us":15814,"threads_started":1}
I20260809 13:12:23.307279 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.274132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:35479 (local address 127.10.181.129:33233)
0809 13:12:23.274381 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.274388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.290383 (+ 15995us) server_negotiation.cc:407] Connection header received
0809 13:12:23.290735 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.290741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.290821 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.290887 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:23.299035 (+ 8148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.300137 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.304133 (+ 3996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.305318 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.305501 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.305578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.306863 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.307022 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:12:23.308455 31797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b416ae5108e459ea6d7ee41ff1cca89" candidate_uuid: "0e5c20fa6d0e4bd39d5120a668756099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58004148e80c4a1ebb4ff41230d3daba" is_pre_election: true
I20260809 13:12:23.308363 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b416ae5108e459ea6d7ee41ff1cca89" candidate_uuid: "0e5c20fa6d0e4bd39d5120a668756099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58004148e80c4a1ebb4ff41230d3daba"
I20260809 13:12:23.309065 31797 raft_consensus.cc:2454] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e5c20fa6d0e4bd39d5120a668756099 in term 0.
I20260809 13:12:23.310981 32342 raft_consensus.cc:686] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 1 LEADER]: Becoming Leader. State: Replica: 0e5c20fa6d0e4bd39d5120a668756099, State: Running, Role: LEADER
I20260809 13:12:23.311501 32342 consensus_queue.cc:237] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } }
I20260809 13:12:23.317827 31559 catalog_manager.cc:5495] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e5c20fa6d0e4bd39d5120a668756099 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "0e5c20fa6d0e4bd39d5120a668756099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5c20fa6d0e4bd39d5120a668756099" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 } health_report { overall_health: UNKNOWN } } }
W20260809 13:12:23.384505 31870 tablet.cc:2369] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:23.398886 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.392911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.10.181.130:36685)
0809 13:12:23.393129 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.393135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.393158 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:12:23.393342 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.393348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.393514 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.393677 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:12:23.394008 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.395082 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.396390 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.397020 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.397145 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.397614 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:12:23.398285 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:12:23.398455 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.398505 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.398587 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 13:12:23.401710 32294 tablet.cc:2369] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:23.408155 32230 raft_consensus.cc:1261] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 1 FOLLOWER]: Refusing update from remote peer 0e5c20fa6d0e4bd39d5120a668756099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:23.408108 31795 raft_consensus.cc:3037] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:23.408973 32342 consensus_queue.cc:1035] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e1c24efdc142789509f4c98ff87362" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:23.411932 31795 raft_consensus.cc:1261] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 1 FOLLOWER]: Refusing update from remote peer 0e5c20fa6d0e4bd39d5120a668756099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:23.412950 32342 consensus_queue.cc:1035] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58004148e80c4a1ebb4ff41230d3daba" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:23.428280 32361 mvcc.cc:205] Tried to move back new op lower bound from 7316607563389968384 to 7316607563008991232. Current Snapshot: MvccSnapshot[applied={T|T < 7316607563389968384}]
I20260809 13:12:23.430740 32364 mvcc.cc:205] Tried to move back new op lower bound from 7316607563389968384 to 7316607563008991232. Current Snapshot: MvccSnapshot[applied={T|T < 7316607563389968384}]
I20260809 13:12:23.457655 31559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:23.457934 31559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:23.461721 31559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f30f43d6c18b42708623e36b52958d6b: Sending DeleteTablet for 3 replicas of tablet 9b416ae5108e459ea6d7ee41ff1cca89
I20260809 13:12:23.462730 31987 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:47552
I20260809 13:12:23.463083 31767 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:54964
I20260809 13:12:23.463428 32190 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b416ae5108e459ea6d7ee41ff1cca89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:40818
I20260809 13:12:23.470695 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 31612
I20260809 13:12:23.472792 32376 tablet_replica.cc:330] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: stopping tablet replica
I20260809 13:12:23.473798 32376 raft_consensus.cc:2229] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:23.474818 32376 raft_consensus.cc:2258] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:23.477779 32376 ts_tablet_manager.cc:1905] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:23.486510 32375 tablet_replica.cc:330] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: stopping tablet replica
I20260809 13:12:23.487119 32375 raft_consensus.cc:2229] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:23.489574 32375 raft_consensus.cc:2258] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:23.491638 32376 ts_tablet_manager.cc:1918] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:23.491825 32376 log.cc:1196] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/9b416ae5108e459ea6d7ee41ff1cca89
I20260809 13:12:23.492158 32375 ts_tablet_manager.cc:1905] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:23.492386 32376 ts_tablet_manager.cc:1939] T 9b416ae5108e459ea6d7ee41ff1cca89 P 52e1c24efdc142789509f4c98ff87362: Deleting consensus metadata
I20260809 13:12:23.494279 31542 catalog_manager.cc:4841] TS 52e1c24efdc142789509f4c98ff87362 (127.10.181.131:34977): tablet 9b416ae5108e459ea6d7ee41ff1cca89 (table test-table [id=9f94dde72df9422390307ba242c48b7a]) successfully deleted
I20260809 13:12:23.501238 32375 ts_tablet_manager.cc:1918] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:23.501549 32375 log.cc:1196] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/9b416ae5108e459ea6d7ee41ff1cca89
I20260809 13:12:23.502003 32374 tablet_replica.cc:330] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: stopping tablet replica
I20260809 13:12:23.502233 32375 ts_tablet_manager.cc:1939] T 9b416ae5108e459ea6d7ee41ff1cca89 P 58004148e80c4a1ebb4ff41230d3daba: Deleting consensus metadata
I20260809 13:12:23.502447 32374 raft_consensus.cc:2229] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:12:23.503080 32374 raft_consensus.cc:2258] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:23.504248 31543 catalog_manager.cc:4841] TS 58004148e80c4a1ebb4ff41230d3daba (127.10.181.129:33233): tablet 9b416ae5108e459ea6d7ee41ff1cca89 (table test-table [id=9f94dde72df9422390307ba242c48b7a]) successfully deleted
I20260809 13:12:23.508328 32374 ts_tablet_manager.cc:1905] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:23.516146 32374 ts_tablet_manager.cc:1918] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:23.516390 32374 log.cc:1196] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/9b416ae5108e459ea6d7ee41ff1cca89
I20260809 13:12:23.516973 32374 ts_tablet_manager.cc:1939] T 9b416ae5108e459ea6d7ee41ff1cca89 P 0e5c20fa6d0e4bd39d5120a668756099: Deleting consensus metadata
I20260809 13:12:23.518826 31542 catalog_manager.cc:4841] TS 0e5c20fa6d0e4bd39d5120a668756099 (127.10.181.130:36685): tablet 9b416ae5108e459ea6d7ee41ff1cca89 (table test-table [id=9f94dde72df9422390307ba242c48b7a]) successfully deleted
I20260809 13:12:23.519954 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.471617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.10.181.129:33233)
0809 13:12:23.471789 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.471795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.498931 (+ 27136us) server_negotiation.cc:407] Connection header received
0809 13:12:23.499431 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.499438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.499528 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.499602 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:23.503489 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.504665 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.506590 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.507202 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.507329 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.511943 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:23.511958 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:23.511977 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:23.512019 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:23.516002 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.517026 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.517037 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.517040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.517217 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.517752 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.517757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.517761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.518363 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:23.518665 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.519306 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.519554 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:12:23.521553 31860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:23.681819 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 31612
I20260809 13:12:23.714802 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 31873
I20260809 13:12:23.733026 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.715750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.10.181.130:36685)
0809 13:12:23.715944 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.715951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.716047 (+ 96us) server_negotiation.cc:407] Connection header received
0809 13:12:23.716257 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.716264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.716357 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.716494 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:23.718310 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.719583 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.721515 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.722170 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.722300 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.725388 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:23.725411 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:23.725430 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:23.725504 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:23.729736 (+ 4232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.730633 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.730642 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.730646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.730839 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.731296 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.731301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.731305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.731770 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:23.732104 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.732385 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.732616 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:12:23.733810 32085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:23.849571 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 31873
I20260809 13:12:23.874605 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 32102
I20260809 13:12:23.887451 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:23.875416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.10.181.131:34977)
0809 13:12:23.875870 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:23.875874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:23.875892 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:12:23.875950 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:23.875954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:23.876016 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:23.876112 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:23.877374 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.878112 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.879357 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:23.879743 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:23.879829 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:23.882314 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:23.882329 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:23.882347 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:23.882379 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:23.885075 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.885694 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.885702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.885704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.885835 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:23.886147 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:23.886150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:23.886152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:23.886513 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:23.886744 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:23.886939 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 13:12:23.887100 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":239,"threads_started":1}
I20260809 13:12:23.888043 32285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:23.974599 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 32102
I20260809 13:12:23.996829 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 31512
I20260809 13:12:23.998006 31601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:24.108022 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 31512
I20260809 13:12:24.138491 10966 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 13:12:24.482726 10966 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 13:12:24.482949 10966 mini_ranger.cc:66] Stopping Ranger...
I20260809 13:12:24.827082 10966 mini_ranger.cc:68] Stopped Ranger
2026-08-09 13:12:24.827 UTC [30037] LOG: received smart shutdown request
2026-08-09 13:12:24.830 UTC [30037] LOG: background worker "logical replication launcher" (PID 30052) exited with exit code 1
2026-08-09 13:12:24.830 UTC [30043] LOG: shutting down
2026-08-09 13:12:24.958 UTC [30037] LOG: database system is shut down
2026-08-09T13:12:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51545 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:12:25 dist-test-slave-42z9 krb5kdc[32400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:12:25 dist-test-slave-42z9 krb5kdc[32400](info): set up 2 sockets
Aug 09 13:12:25 dist-test-slave-42z9 krb5kdc[32400](info): commencing operation
krb5kdc: starting...
W20260809 13:12:26.266958 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:12:26 dist-test-slave-42z9 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:12:26.534952 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43231
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:26.866712 32424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:26.866991 32424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:26.867123 32424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:26.874013 32424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:26.874120 32424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:26.874198 32424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:26.874276 32424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:26.874336 32424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:26.883752 32424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:43231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:26.885000 32424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:26.886260 32424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:26.897271 32434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:26.897353 32435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:26.897228 32437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:26.898524 32424 server_base.cc:1034] running on GCE node
I20260809 13:12:26.899241 32424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:26.900520 32424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:26.901712 32424 hybrid_clock.cc:648] HybridClock initialized: now 1786281146901667 us; error 34 us; skew 500 ppm
I20260809 13:12:26.902027 32424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:26 dist-test-slave-42z9 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:26.904891 32424 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:26.906229 32424 webserver.cc:460] Webserver started at http://127.10.181.190:38681/ using document root <none> and password file <none>
I20260809 13:12:26.906869 32424 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:26.906984 32424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:26.907289 32424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:26.910238 32424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "af7f082e3fc04295947bba81132c83f0"
format_stamp: "Formatted at 2026-08-09 13:12:26 on dist-test-slave-42z9"
server_key: "5326054f649ad1275fecca280865ae92"
server_key_iv: "a82bbbe49f82ef94e52b1c93ef8203ca"
server_key_version: "encryptionkey@0"
I20260809 13:12:26.911001 32424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "af7f082e3fc04295947bba81132c83f0"
format_stamp: "Formatted at 2026-08-09 13:12:26 on dist-test-slave-42z9"
server_key: "5326054f649ad1275fecca280865ae92"
server_key_iv: "a82bbbe49f82ef94e52b1c93ef8203ca"
server_key_version: "encryptionkey@0"
I20260809 13:12:26.915728 32424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 13:12:26.919024 32446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:26.920218 32424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:12:26.920374 32424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "af7f082e3fc04295947bba81132c83f0"
format_stamp: "Formatted at 2026-08-09 13:12:26 on dist-test-slave-42z9"
server_key: "5326054f649ad1275fecca280865ae92"
server_key_iv: "a82bbbe49f82ef94e52b1c93ef8203ca"
server_key_version: "encryptionkey@0"
I20260809 13:12:26.920539 32424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:26.945650 32424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:26.949397 32424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:26.949688 32424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:26.968679 32424 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:43231
I20260809 13:12:26.968721 32521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:43231 every 8 connection(s)
I20260809 13:12:26.970026 32424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:26.973590 32522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:26.979892 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 32424
I20260809 13:12:26.980132 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:12:26.980062 32522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:12:26.980969 10966 external_mini_cluster.cc:1428] Setting key 790c2f654eb0fb0d75c6e002224f84b8
Aug 09 13:12:26 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:27.002851 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:26.983642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.10.181.190:43231)
0809 13:12:26.985435 (+ 1793us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:26.985467 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:26.985522 (+ 55us) server_negotiation.cc:407] Connection header received
0809 13:12:26.987308 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:26.987314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:26.987523 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:26.987682 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:26.989120 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:26.990333 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:26.992096 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:26.992628 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:26.992751 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:26.996281 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:26.996289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:26.996300 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:26.996329 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:26.998855 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:26.999644 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:26.999650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:26.999653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:26.999791 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.000240 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.000244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.000247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.000712 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.000872 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.001526 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.001788 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1606,"spinlock_wait_cycles":22912,"thread_start_us":1388,"threads_started":1}
I20260809 13:12:27.002876 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:26.983642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43231 (local address 127.0.0.1:45168)
0809 13:12:26.984490 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:26.984551 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:26.986949 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:26.987772 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:26.987779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:26.988482 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:26.988931 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:26.988954 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:26.990487 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:26.990492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:26.991971 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:26.991985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:26.992863 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:26.992869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:26.993050 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:26.993758 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:26.993781 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:26.996059 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:26.998990 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:26.999002 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:26.999014 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:26.999525 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:26.999898 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:26.999902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:26.999905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.000105 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.000858 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:27.000866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:27.001096 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 13:12:27.001362 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 13:12:27.001947 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":82,"thread_start_us":364,"threads_started":1}
I20260809 13:12:27.005864 32522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:12:27.006788 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.006443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43926)
0809 13:12:27.006590 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.006691 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 13:12:27.007001 32449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
I20260809 13:12:27.008486 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.008149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43928)
0809 13:12:27.008304 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.008369 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 13:12:27.008682 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:37611: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.181.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
I20260809 13:12:27.042361 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:37611
--webserver_interface=127.10.181.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:27.058008 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } attempt: 1
I20260809 13:12:27.059159 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.058679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43936)
0809 13:12:27.058887 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.059004 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 13:12:27.060199 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.059842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43950)
0809 13:12:27.059992 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.060057 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 13:12:27.060402 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:37611: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
I20260809 13:12:27.128624 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } attempt: 2
I20260809 13:12:27.129758 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.129248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43964)
0809 13:12:27.129522 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.129613 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:12:27.130785 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.130419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43968)
0809 13:12:27.130566 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.130649 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 13:12:27.130976 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:37611: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
I20260809 13:12:27.222211 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } attempt: 3
I20260809 13:12:27.223371 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.222883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43978)
0809 13:12:27.223103 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.223233 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 13:12:27.224442 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.224061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43990)
0809 13:12:27.224234 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.224303 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 13:12:27.224644 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:37611: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
I20260809 13:12:27.367862 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } attempt: 4
I20260809 13:12:27.369014 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.368480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43992)
0809 13:12:27.368790 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.368873 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 13:12:27.370056 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.369704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:43994)
0809 13:12:27.369851 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.369937 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 13:12:27.370249 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:37611: Network error: Client connection negotiation failed: client connection to 127.10.181.189:37611: connect: Connection refused (error 111)
W20260809 13:12:27.374612 32535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:27.374882 32535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:27.375006 32535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:27.382006 32535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:27.382117 32535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:27.382189 32535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:27.382279 32535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:27.382337 32535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:27.391816 32535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:37611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.181.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:27.393013 32535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:27.394308 32535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:27.405293 32543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:27.405391 32545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:27.405423 32547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:27.406455 32535 server_base.cc:1034] running on GCE node
I20260809 13:12:27.407166 32535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:27.408413 32535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:27.409603 32535 hybrid_clock.cc:648] HybridClock initialized: now 1786281147409554 us; error 40 us; skew 500 ppm
I20260809 13:12:27.409927 32535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:27.415912 32535 init.cc:377] Logged in from keytab as kudu/127.10.181.189@KRBTEST.COM (short username kudu)
I20260809 13:12:27.417347 32535 webserver.cc:460] Webserver started at http://127.10.181.189:41279/ using document root <none> and password file <none>
I20260809 13:12:27.418079 32535 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:27.418196 32535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:27.418488 32535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:27.421396 32535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data/instance:
uuid: "ce4a736826084090bce10882c2cf6251"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "e85dd8d00540bbec23604bfe6836a0fa"
server_key_iv: "b97855296595ba7c83bb17fbb27fd705"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.422246 32535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal/instance:
uuid: "ce4a736826084090bce10882c2cf6251"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "e85dd8d00540bbec23604bfe6836a0fa"
server_key_iv: "b97855296595ba7c83bb17fbb27fd705"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.427395 32535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:12:27.430755 32562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:27.431970 32535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:12:27.432137 32535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "ce4a736826084090bce10882c2cf6251"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "e85dd8d00540bbec23604bfe6836a0fa"
server_key_iv: "b97855296595ba7c83bb17fbb27fd705"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.432304 32535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:27.442835 32535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:27.446393 32535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:27.446655 32535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:27.463922 32535 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.189:37611
I20260809 13:12:27.463963 32634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.189:37611 every 8 connection(s)
I20260809 13:12:27.465270 32535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
I20260809 13:12:27.468338 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 32535
I20260809 13:12:27.468521 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-1/wal/instance
I20260809 13:12:27.468921 32635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:27.468955 10966 external_mini_cluster.cc:1428] Setting key c277f2fa2f6a91c6094a61d4421c8ad0
I20260809 13:12:27.475212 32635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.181.188@KRBTEST.COM; defaulting to no policy
I20260809 13:12:27.494822 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.477722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.10.181.190:43231)
0809 13:12:27.477897 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:27.477901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:27.479099 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 13:12:27.481177 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:27.481182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:27.481256 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:27.481338 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:27.482988 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.483672 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.485221 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.485821 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.485959 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.489264 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:27.489269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:27.489273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:27.489306 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:27.491842 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.492558 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.492562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.492565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.492705 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.493111 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.493123 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.493125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.493404 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.493594 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.494267 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.494501 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:12:27.495875 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.478203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43231 (local address 127.0.0.1:45176)
0809 13:12:27.478933 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.478970 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:27.480864 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:27.481639 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:27.481651 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:27.482234 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:27.482808 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.482826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.483833 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.483837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.485071 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.485084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.486084 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.486092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.486318 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.486931 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:27.486953 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:27.489039 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:27.491975 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.491984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.492000 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.492419 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.492816 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.492821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.492825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.493000 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.493546 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:27.493553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:27.493900 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 13:12:27.494145 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 13:12:27.494642 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":311,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.188@KRBTEST.COM" created.
I20260809 13:12:27.497587 32635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:12:27.511920 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.498264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:44002)
0809 13:12:27.498416 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.498431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:27.498510 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:27.499433 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:27.499437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:27.499845 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:27.499971 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.499979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.501289 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.501293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.502154 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.502166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.503140 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.503150 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.503341 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.503923 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:27.503944 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:27.506254 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:27.509566 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.509579 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.509584 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.509924 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.510276 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.510281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.510285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.510423 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.510986 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:27.510991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:27.511127 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 13:12:27.511242 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 13:12:27.511503 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:12:27.511991 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.498518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44002 (local address 127.10.181.189:37611)
0809 13:12:27.499028 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:27.499034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:27.499059 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:12:27.499100 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:27.499106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:27.499275 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:27.499418 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:27.500118 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.501156 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.502287 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.502977 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.503084 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.506489 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:27.506503 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:27.506521 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:27.506561 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:27.509424 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.510044 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.510053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.510057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.510175 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.510518 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.510523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.510527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.510861 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.511048 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.511402 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.511605 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260809 13:12:27.514544 32635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:12:27.515575 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.515145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54062)
0809 13:12:27.515304 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.515422 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 13:12:27.515801 32565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.517331 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.517000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54064)
0809 13:12:27.517156 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.517230 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 13:12:27.517602 32635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
I20260809 13:12:27.529671 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:39493
--webserver_interface=127.10.181.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:27.566892 32635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 1
I20260809 13:12:27.568113 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.567650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54068)
0809 13:12:27.567844 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.567949 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:12:27.569026 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.568704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54072)
0809 13:12:27.568858 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.568923 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 13:12:27.569227 32635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.637410 32635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 2
I20260809 13:12:27.638474 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.638027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54076)
0809 13:12:27.638224 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.638314 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 13:12:27.639418 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.639075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54084)
0809 13:12:27.639223 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.639288 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 13:12:27.639657 32635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.669540 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } attempt: 5
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:12:27.684158 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.670274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:44004)
0809 13:12:27.670519 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.670541 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:27.670687 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:27.671039 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:27.671045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:27.671554 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:27.671817 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.671831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.672808 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.672814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.674288 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.674306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.674976 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.674982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.675184 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.675792 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:27.675815 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:27.678273 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:27.681940 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.681944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.681947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.682298 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.682723 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.682729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.682732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.682888 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.683481 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:27.683484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:27.683619 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 13:12:27.683731 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 13:12:27.683924 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 13:12:27.684398 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.670502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.10.181.189:37611)
0809 13:12:27.670664 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:27.670668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:27.670688 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:12:27.670805 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:27.670809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:27.670890 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:27.670969 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:27.671962 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.672667 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.674420 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.674824 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.674912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.678481 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:27.678489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:27.678494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:27.678536 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:27.681803 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.682419 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.682426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.682430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.682593 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.682994 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.682998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.683001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.683376 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.683534 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.683851 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.684078 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:12:27.685427 32522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:12:27.686525 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.686094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54100)
0809 13:12:27.686269 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.686363 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 13:12:27.687491 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.687123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54108)
0809 13:12:27.687269 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.687356 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 13:12:27.687733 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.730862 32635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 3
I20260809 13:12:27.731925 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.731476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54120)
0809 13:12:27.731671 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.731762 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:12:27.733162 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.732687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54134)
0809 13:12:27.732925 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.733001 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260809 13:12:27.733470 32635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.738976 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 1
I20260809 13:12:27.740104 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.739602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54148)
0809 13:12:27.739806 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.739916 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:12:27.741119 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.740752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54164)
0809 13:12:27.740897 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.740986 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 13:12:27.741345 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.809538 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 2
I20260809 13:12:27.810731 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.810186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54168)
0809 13:12:27.810405 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.810519 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 13:12:27.811756 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.811356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54184)
0809 13:12:27.811506 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.811611 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 13:12:27.811983 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
W20260809 13:12:27.875058 32645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:27.875341 32645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:27.875447 32645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 13:12:27.876749 32635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 4
I20260809 13:12:27.877936 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.877416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54192)
0809 13:12:27.877684 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.877775 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 13:12:27.878933 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.878610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54208)
0809 13:12:27.878750 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.878824 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 13:12:27.879133 32635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
W20260809 13:12:27.883046 32645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:27.883145 32645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:27.883224 32645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:27.883297 32645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:27.883355 32645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:27.893098 32645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:43231,127.10.181.189:37611,127.10.181.188:39493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.181.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:27.894459 32645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:27.895735 32645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:27.906997 32656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:27.907071 32657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:27.907059 32660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:27.907894 32645 server_base.cc:1034] running on GCE node
I20260809 13:12:27.908617 32645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:27.909916 32645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:27.911121 32645 hybrid_clock.cc:648] HybridClock initialized: now 1786281147911059 us; error 48 us; skew 500 ppm
I20260809 13:12:27.911454 32645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:27.914388 32645 init.cc:377] Logged in from keytab as kudu/127.10.181.188@KRBTEST.COM (short username kudu)
I20260809 13:12:27.915833 32645 webserver.cc:460] Webserver started at http://127.10.181.188:35877/ using document root <none> and password file <none>
I20260809 13:12:27.916479 32645 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:27.916589 32645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:27.916891 32645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:27.918208 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 3
I20260809 13:12:27.919324 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.918781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54224)
0809 13:12:27.919021 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.919132 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:12:27.920316 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.919939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54228)
0809 13:12:27.920089 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.920187 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:12:27.920547 32522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:39493: Network error: Client connection negotiation failed: client connection to 127.10.181.188:39493: connect: Connection refused (error 111)
I20260809 13:12:27.920444 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data/instance:
uuid: "eb95b199853f4643b490a275b97b0262"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "f0e0e7d4e455efc315763d70605e1113"
server_key_iv: "17f72ddb933caf0ecd11a98aea45166a"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.921254 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal/instance:
uuid: "eb95b199853f4643b490a275b97b0262"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "f0e0e7d4e455efc315763d70605e1113"
server_key_iv: "17f72ddb933caf0ecd11a98aea45166a"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.926267 32645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 13:12:27.929533 32672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:27.930877 32645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:12:27.931046 32645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "eb95b199853f4643b490a275b97b0262"
format_stamp: "Formatted at 2026-08-09 13:12:27 on dist-test-slave-42z9"
server_key: "f0e0e7d4e455efc315763d70605e1113"
server_key_iv: "17f72ddb933caf0ecd11a98aea45166a"
server_key_version: "encryptionkey@0"
I20260809 13:12:27.931226 32645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:27.948148 32645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:27.951821 32645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:27.952103 32645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:27.969511 32645 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.188:39493
I20260809 13:12:27.969529 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.188:39493 every 8 connection(s)
I20260809 13:12:27.970870 32645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
I20260809 13:12:27.975345 32763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:27.976608 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 32645
I20260809 13:12:27.976794 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786280946828509-10966-0/minicluster-data/master-2/wal/instance
I20260809 13:12:27.977248 10966 external_mini_cluster.cc:1428] Setting key dacacdfece7fc5e93f5c175a4a743b39
I20260809 13:12:27.983973 32763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:12:27 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:27.998767 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.979126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.10.181.188:39493)
0809 13:12:27.979909 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:27.979921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:27.979965 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:12:27.981667 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:27.981691 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:27.982070 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:27.982436 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:27.983891 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.985219 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.986820 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.987557 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.987715 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.992047 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:27.992057 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:27.992068 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:27.992120 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:27.994999 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.995615 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.995622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.995628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.995772 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.996142 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.996146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.996148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.996634 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.996913 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.997362 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.997827 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":344,"threads_started":1}
I20260809 13:12:28.000006 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.986396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.10.181.190:43231)
0809 13:12:27.986616 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:27.986621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:27.987033 (+ 412us) server_negotiation.cc:407] Connection header received
0809 13:12:27.987208 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:27.987212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:27.987278 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:27.987360 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:27.988790 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.989536 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.991160 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.991576 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.991687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.994933 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:27.994939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:27.994942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:27.994974 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:27.997377 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.998207 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.998212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.998215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.998342 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:27.998701 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:27.998705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:27.998707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:27.998969 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:27.999106 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:27.999600 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 13:12:27.999760 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 13:12:28.000309 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:27.986211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:43231 (local address 127.0.0.1:45192)
0809 13:12:27.986907 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:27.986942 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:27.987085 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:27.987454 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:27.987461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:27.988293 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:27.988629 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.988644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.989691 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.989697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.991007 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:27.991020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:27.991796 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:27.991803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:27.991954 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:27.992550 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:27.992568 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:27.994745 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:27.997525 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.997537 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.997544 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.998082 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.998453 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:27.998458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:27.998463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:27.998591 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:27.999098 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:27.999105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:27.999325 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 13:12:27.999535 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 13:12:27.999830 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":281,"threads_started":1}
I20260809 13:12:28.001880 32763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:28.014623 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.002515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:37611 (local address 127.0.0.1:44006)
0809 13:12:28.002647 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:28.002663 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:28.002754 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:28.003095 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:28.003099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:28.003429 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:28.003558 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.003566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.004481 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.004486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.005330 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.005342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.006053 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.006059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.006248 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.006720 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:28.006743 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:28.008763 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:28.012220 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.012224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.012226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.012568 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.012962 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.012964 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.012967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.013090 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.013693 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:28.013698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:28.013871 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 13:12:28.014076 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 13:12:28.014303 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 13:12:28.014667 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.002647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.10.181.189:37611)
0809 13:12:28.002842 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.002848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.002865 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:12:28.002914 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.002917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.002972 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.003070 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.003706 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.004352 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.005499 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.005918 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.006027 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.008964 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.008971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.008976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.009018 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.012093 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.012695 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.012701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.012705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.012860 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.013184 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.013189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.013192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.013563 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.013693 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.014159 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.014377 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:12:28.015643 32763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:12:28.016281 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.001909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.10.181.190:43231)
0809 13:12:28.002061 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.002066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.002093 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:12:28.002246 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.002249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.002308 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.002404 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.003657 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.004351 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.005672 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.006057 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.006173 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.010406 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.010413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.010418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.010456 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.013296 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.014141 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.014146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.014149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.014301 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.014738 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.014742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.014745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.015080 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.015327 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.015773 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.015983 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:12:28.030436 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.016330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54250)
0809 13:12:28.017146 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:28.017161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:28.017258 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:28.017591 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:28.017596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:28.017989 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:28.018161 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.018170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.019473 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.019477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.020667 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.020682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.021533 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.021537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.021689 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.022184 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:28.022200 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:28.024631 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:28.028101 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.028106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.028109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.028553 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.028932 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.028937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.028940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.029110 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.029695 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:28.029698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:28.029830 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 13:12:28.029953 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 13:12:28.030165 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715}
I20260809 13:12:28.030840 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.016555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.10.181.188:39493)
0809 13:12:28.016717 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.016722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.017228 (+ 506us) server_negotiation.cc:407] Connection header received
0809 13:12:28.017349 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.017352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.017407 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.017567 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.018317 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.019332 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.020810 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.021351 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.021517 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.024830 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.024836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.024841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.024880 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.027984 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.028670 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.028676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.028679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.028826 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.029235 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.029238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.029240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.029573 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.029698 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.030086 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.030313 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:12:28.034057 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.017911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.10.181.189:37611)
0809 13:12:28.018066 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.018071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.018088 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:12:28.018249 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.018254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.018325 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.018401 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.019862 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.020902 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.022348 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.022989 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.023112 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.027978 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.027983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.027986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.028016 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.031194 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.031969 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.031975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.031978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.032188 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.032540 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.032545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.032548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.032870 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.033158 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.033337 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.033581 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:12:28.036919 32763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: Bootstrap starting.
I20260809 13:12:28.038908 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:12:28.040356 32763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:28.041347 32763 log.cc:824] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:28.044098 32763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: No bootstrap required, opened a new log
I20260809 13:12:28.054666 32763 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.055106 32763 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:28.055217 32763 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb95b199853f4643b490a275b97b0262, State: Initialized, Role: FOLLOWER
I20260809 13:12:28.056450 32763 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.063278 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.043497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.10.181.190:43231)
0809 13:12:28.043693 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.043698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.044062 (+ 364us) server_negotiation.cc:407] Connection header received
0809 13:12:28.044365 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.044372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.044462 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.044533 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.047207 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.048388 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.050770 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.051505 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.051605 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.055899 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.055906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.055910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.055952 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.059767 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.060753 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.060759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.060762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.060939 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.061416 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.061422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.061425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.061835 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.062056 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.062667 (+ 611us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.062904 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:12:28.064042 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.044204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.10.181.189:37611)
0809 13:12:28.044390 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.044396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.045014 (+ 618us) server_negotiation.cc:407] Connection header received
0809 13:12:28.045275 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.045280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.045354 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.045422 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.047692 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.048752 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.050996 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.051642 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.051740 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.056123 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.056130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.056134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.056170 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.059531 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.060617 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.060624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.060628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.060807 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.061280 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.061285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.061289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.061695 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.061953 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.063176 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.063660 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 13:12:28.066040 319 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.066586 319 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.067381 32763 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:28.070676 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.045330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.10.181.188:39493)
0809 13:12:28.045555 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.045561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.055893 (+ 10332us) server_negotiation.cc:407] Connection header received
0809 13:12:28.056057 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.056063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.056142 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.056249 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.057831 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.058942 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.060902 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.061638 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.061731 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.064846 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.064852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.064857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.064894 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.067768 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.068504 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.068509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.068513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.068698 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.069149 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.069154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.069160 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.069602 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.069871 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.070156 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.070367 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 13:12:28.080367 337 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:12:28.080552 337 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:12:28.077634 32645 master_runner.cc:186] Error getting master registration for 127.10.181.190:43231: OK, instance_id { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" instance_seqno: 1786281146963826 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:12:28.085752 32645 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:12:28.097738 32522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 4
I20260809 13:12:28.100956 319 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:12:28.101341 319 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.103019 319 leader_election.cc:288] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af7f082e3fc04295947bba81132c83f0 (127.10.181.190:43231), ce4a736826084090bce10882c2cf6251 (127.10.181.189:37611)
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281146, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
I20260809 13:12:28.115116 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.098406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.10.181.188:39493)
0809 13:12:28.098547 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.098551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.098566 (+ 15us) server_negotiation.cc:407] Connection header received
0809 13:12:28.098614 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.098617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.098672 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.098742 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.099686 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.100711 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.103048 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.103743 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.103857 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.107888 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.107895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.107900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.107944 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.111226 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.112033 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.112039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.112043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.112229 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.112684 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.112692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.112695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.113354 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.113637 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.114512 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.114746 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 13:12:28.115451 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.098183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54272)
0809 13:12:28.098383 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:28.098405 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:28.098512 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:28.098799 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:28.098804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:28.099303 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:28.099543 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.099556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.101383 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.101389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.102859 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.102876 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.104055 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.104061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.104258 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.104894 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:28.104918 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:28.107703 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:28.111352 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.111358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.111363 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.111907 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.112352 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.112357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.112361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.112519 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.113762 (+ 1243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:28.113768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:28.114099 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 13:12:28.114416 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 13:12:28.114885 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:12:28.121510 32522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: Bootstrap starting.
I20260809 13:12:28.125735 32522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:28.127099 32522 log.cc:824] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:28.130452 32522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: No bootstrap required, opened a new log
I20260809 13:12:28.141580 32522 raft_consensus.cc:348] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.141963 32522 raft_consensus.cc:374] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:28.142079 32522 raft_consensus.cc:729] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7f082e3fc04295947bba81132c83f0, State: Initialized, Role: FOLLOWER
I20260809 13:12:28.142716 32522 consensus_queue.cc:260] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.143702 342 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.144079 342 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.145037 32522 sys_catalog.cc:564] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:28.145061 32486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb95b199853f4643b490a275b97b0262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af7f082e3fc04295947bba81132c83f0" is_pre_election: true
I20260809 13:12:28.145694 32486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb95b199853f4643b490a275b97b0262 in term 0.
I20260809 13:12:28.146463 32675 leader_election.cc:302] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af7f082e3fc04295947bba81132c83f0, eb95b199853f4643b490a275b97b0262; no voters:
I20260809 13:12:28.147045 319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:12:28.147218 319 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:12:28.147334 319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:28.151445 319 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.152248 319 leader_election.cc:288] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [CANDIDATE]: Term 1 election: Requested vote from peers af7f082e3fc04295947bba81132c83f0 (127.10.181.190:43231), ce4a736826084090bce10882c2cf6251 (127.10.181.189:37611)
I20260809 13:12:28.153414 32486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb95b199853f4643b490a275b97b0262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af7f082e3fc04295947bba81132c83f0"
I20260809 13:12:28.153640 32486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:28.156816 32486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb95b199853f4643b490a275b97b0262 in term 1.
I20260809 13:12:28.157210 32675 leader_election.cc:302] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af7f082e3fc04295947bba81132c83f0, eb95b199853f4643b490a275b97b0262; no voters:
I20260809 13:12:28.157519 319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:28.172613 319 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [term 1 LEADER]: Becoming Leader. State: Replica: eb95b199853f4643b490a275b97b0262, State: Running, Role: LEADER
I20260809 13:12:28.173801 319 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.178321 32635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } attempt: 5
I20260809 13:12:28.181670 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb95b199853f4643b490a275b97b0262. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.182122 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:28.174006 32424 master_runner.cc:186] Error getting master registration for 127.10.181.189:37611: OK, instance_id { permanent_uuid: "ce4a736826084090bce10882c2cf6251" instance_seqno: 1786281147458999 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:12:28.184417 32424 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 13:12:28 dist-test-slave-42z9 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281147, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
W20260809 13:12:28.202340 32457 tablet.cc:2369] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:12:28.204046 32685 tablet.cc:2369] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:28.207000 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.178923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:39493 (local address 127.0.0.1:54274)
0809 13:12:28.179297 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:28.179318 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:28.179447 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:28.180324 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:28.180330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:28.180857 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:28.181086 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.181100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.182767 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.182774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.184275 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:28.184292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.185678 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.185685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:28.186098 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.187054 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:28.187111 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:28.191589 (+ 4478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:28.198654 (+ 7065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.198783 (+ 129us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.198788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.199686 (+ 898us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.200532 (+ 846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:28.200537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:28.200570 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:28.200914 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:28.202277 (+ 1363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:28.202311 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:28.202590 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 13:12:28.202845 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 13:12:28.203394 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260809 13:12:28.210053 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:28.179674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.10.181.188:39493)
0809 13:12:28.179837 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:28.179842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:28.179859 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:12:28.179911 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:28.179915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:28.179993 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:28.180103 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:28.181299 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.182451 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.184504 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:28.185228 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:28.185368 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:28.191909 (+ 6541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:28.191916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:28.191921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:28.191962 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:28.198412 (+ 6450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.199975 (+ 1563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.200014 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.200018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.200313 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:28.201193 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:28.201198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:28.201202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:28.202041 (+ 839us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:28.205025 (+ 2984us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:28.206242 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0809 13:12:28.208289 (+ 2047us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:12:28.216606 32635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: Bootstrap starting.
I20260809 13:12:28.228012 32635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: Neither blocks nor log segments found. Creating new log.
W20260809 13:12:28.228797 361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:12:28.229723 361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:12:28.230276 365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:28.232501 365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:28.235177 32635 log.cc:824] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:28.241093 32635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: No bootstrap required, opened a new log
I20260809 13:12:28.250828 32486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [term 1 FOLLOWER]: Refusing update from remote peer eb95b199853f4643b490a275b97b0262: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:28.251845 356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:28.254902 32635 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.255581 32635 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:28.255803 32635 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4a736826084090bce10882c2cf6251, State: Initialized, Role: FOLLOWER
I20260809 13:12:28.256757 32635 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } }
I20260809 13:12:28.264837 32600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb95b199853f4643b490a275b97b0262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4a736826084090bce10882c2cf6251"
I20260809 13:12:28.265151 32599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:28.265887 32603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb95b199853f4643b490a275b97b0262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4a736826084090bce10882c2cf6251" is_pre_election: true
I20260809 13:12:28.269009 32599 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [term 1 FOLLOWER]: Refusing update from remote peer eb95b199853f4643b490a275b97b0262: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:28.269506 32635 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:28.269820 356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:28.270991 372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.271384 372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.290904 342 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb95b199853f4643b490a275b97b0262. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.291276 342 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.295627 367 mvcc.cc:205] Tried to move back new op lower bound from 7316607583217635328 to 7316607582925557760. Current Snapshot: MvccSnapshot[applied={T|T < 7316607583217635328}]
I20260809 13:12:28.296065 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.296330 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:28.296586 370 mvcc.cc:205] Tried to move back new op lower bound from 7316607583217635328 to 7316607582925557760. Current Snapshot: MvccSnapshot[applied={T|T < 7316607583217635328}]
I20260809 13:12:28.297611 319 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.297704 372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb95b199853f4643b490a275b97b0262. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.297924 319 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:28.297960 372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.300527 365 catalog_manager.cc:1347] Generated new cluster ID: d3a7a5ac39e642e19fa24ec24aff511d
I20260809 13:12:28.300626 365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:28.307351 376 mvcc.cc:205] Tried to move back new op lower bound from 7316607583217635328 to 7316607582925557760. Current Snapshot: MvccSnapshot[applied={T|T < 7316607583217635328}]
I20260809 13:12:28.311112 372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.311334 372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.316948 342 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb95b199853f4643b490a275b97b0262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af7f082e3fc04295947bba81132c83f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 43231 } } peers { permanent_uuid: "ce4a736826084090bce10882c2cf6251" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 37611 } } peers { permanent_uuid: "eb95b199853f4643b490a275b97b0262" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 39493 } } }
I20260809 13:12:28.317174 342 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:12:28.326452 365 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:12:28.328352 365 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:28.348150 397 catalog_manager.cc:1259] Loaded cluster ID: d3a7a5ac39e642e19fa24ec24aff511d
I20260809 13:12:28.348255 397 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: loading cluster ID for follower catalog manager: success
I20260809 13:12:28.351926 397 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: acquiring CA information for follower catalog manager: success
W20260809 13:12:28.353102 397 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260809 13:12:28.364187 365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb95b199853f4643b490a275b97b0262: Generated new TSK 0
I20260809 13:12:28.365315 365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:12:29.231747 361 catalog_manager.cc:1259] Loaded cluster ID: d3a7a5ac39e642e19fa24ec24aff511d
I20260809 13:12:29.231851 361 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: loading cluster ID for follower catalog manager: success
I20260809 13:12:29.234416 361 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: acquiring CA information for follower catalog manager: success
I20260809 13:12:29.236420 361 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af7f082e3fc04295947bba81132c83f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:12:29.355839 397 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce4a736826084090bce10882c2cf6251: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:12:29.511059 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 32424
I20260809 13:12:29.512519 32516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:29.622319 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 32424
I20260809 13:12:29.652011 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 32535
I20260809 13:12:29.653071 32629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:29.759097 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 32535
I20260809 13:12:29.785360 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 32645
I20260809 13:12:29.786435 32751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:29.894801 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 32645
2026-08-09T13:12:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4839 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:12:30 dist-test-slave-42z9 krb5kdc[418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:12:30 dist-test-slave-42z9 krb5kdc[418](info): set up 2 sockets
Aug 09 13:12:30 dist-test-slave-42z9 krb5kdc[418](info): commencing operation
krb5kdc: starting...
W20260809 13:12:31.117362 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:12:31 dist-test-slave-42z9 krb5kdc[418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:12:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:12:31.390570 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:45873
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:45873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:31.723318 439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:31.723596 439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:31.723691 439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:31.730794 439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:31.730921 439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:31.730998 439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:31.731081 439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:31.731141 439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:31.740646 439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:45873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:45873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:31.741932 439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:31.743201 439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:31.754613 447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:31.754650 445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:31.754624 451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:31.755434 439 server_base.cc:1034] running on GCE node
I20260809 13:12:31.756104 439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:31.757395 439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:31.758625 439 hybrid_clock.cc:648] HybridClock initialized: now 1786281151758573 us; error 38 us; skew 500 ppm
I20260809 13:12:31.758942 439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:31 dist-test-slave-42z9 krb5kdc[418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281151, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:31.761843 439 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:31.763271 439 webserver.cc:460] Webserver started at http://127.10.181.190:33601/ using document root <none> and password file <none>
I20260809 13:12:31.763945 439 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:31.764061 439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:31.764349 439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:31.767221 439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "6fb79c43772f47ee8f5bdb095aae9fb9"
format_stamp: "Formatted at 2026-08-09 13:12:31 on dist-test-slave-42z9"
server_key: "97ee0a127af425f7cc1f5ad0996c0613"
server_key_iv: "be9b8149da66bc0b39dc5db1c89ab687"
server_key_version: "encryptionkey@0"
I20260809 13:12:31.767932 439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "6fb79c43772f47ee8f5bdb095aae9fb9"
format_stamp: "Formatted at 2026-08-09 13:12:31 on dist-test-slave-42z9"
server_key: "97ee0a127af425f7cc1f5ad0996c0613"
server_key_iv: "be9b8149da66bc0b39dc5db1c89ab687"
server_key_version: "encryptionkey@0"
I20260809 13:12:31.772845 439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:12:31.776136 467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:31.777370 439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:12:31.777585 439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "6fb79c43772f47ee8f5bdb095aae9fb9"
format_stamp: "Formatted at 2026-08-09 13:12:31 on dist-test-slave-42z9"
server_key: "97ee0a127af425f7cc1f5ad0996c0613"
server_key_iv: "be9b8149da66bc0b39dc5db1c89ab687"
server_key_version: "encryptionkey@0"
I20260809 13:12:31.777755 439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:31.800870 439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:31.804874 439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:31.805154 439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:31.821218 439 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:45873
I20260809 13:12:31.821228 552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:45873 every 8 connection(s)
I20260809 13:12:31.822494 439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:31.826154 553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:31.827728 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 439
I20260809 13:12:31.827961 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:12:31.828779 10966 external_mini_cluster.cc:1428] Setting key bdc4203850de0fdde63570fab3462c39
I20260809 13:12:31.840054 553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9: Bootstrap starting.
Aug 09 13:12:31 dist-test-slave-42z9 krb5kdc[418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:31.843698 553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:31.844707 553 log.cc:824] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:31.847498 553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9: No bootstrap required, opened a new log
I20260809 13:12:31.851203 556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:31.830567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.10.181.190:45873)
0809 13:12:31.831327 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:31.831341 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:31.831387 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:12:31.832936 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:31.832961 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:31.833327 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:31.833680 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:31.835476 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:31.836790 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:31.838737 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:31.839412 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:31.839545 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:31.844173 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:31.844181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:31.844195 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:31.844235 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:31.847238 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:31.847941 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:31.847948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:31.847954 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:31.848137 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:31.848516 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:31.848520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:31.848522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:31.849031 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:31.849342 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:31.849847 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 13:12:31.850248 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":325,"threads_started":1}
I20260809 13:12:31.855007 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:12:31.855649 553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } }
I20260809 13:12:31.855968 553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:31.856052 553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb79c43772f47ee8f5bdb095aae9fb9, State: Initialized, Role: FOLLOWER
I20260809 13:12:31.856577 553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } }
I20260809 13:12:31.856796 553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:31.856916 553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:31.857061 553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:31.859472 553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } }
I20260809 13:12:31.859951 553 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb79c43772f47ee8f5bdb095aae9fb9; no voters:
I20260809 13:12:31.860478 553 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:12:31.860605 559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:31.861279 559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 6fb79c43772f47ee8f5bdb095aae9fb9, State: Running, Role: LEADER
I20260809 13:12:31.861788 559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } }
I20260809 13:12:31.862278 553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:31.864625 560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } } }
I20260809 13:12:31.864987 560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:31.866569 570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:31.866482 561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb79c43772f47ee8f5bdb095aae9fb9. Latest consensus state: current_term: 1 leader_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb79c43772f47ee8f5bdb095aae9fb9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 45873 } } }
I20260809 13:12:31.866787 561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:31.870755 570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:31.879076 570 catalog_manager.cc:1347] Generated new cluster ID: fcbb9610b8344919ba7224184655e5ee
I20260809 13:12:31.879181 570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:31.915428 570 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:12:31.916712 570 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:31.932732 570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fb79c43772f47ee8f5bdb095aae9fb9: Generated new TSK 0
I20260809 13:12:31.933534 570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:12:31.967427 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 439
I20260809 13:12:31.968505 545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:32.063446 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 439
2026-08-09T13:12:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2165 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:12:32 dist-test-slave-42z9 krb5kdc[596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:12:32 dist-test-slave-42z9 krb5kdc[596](info): set up 2 sockets
Aug 09 13:12:32 dist-test-slave-42z9 krb5kdc[596](info): commencing operation
krb5kdc: starting...
W20260809 13:12:33.286525 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:12:33 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:12:33.551970 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:33.880048 618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:33.880332 618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:33.880462 618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:33.887615 618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:33.887723 618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:33.887794 618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:33.887871 618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:33.887933 618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:33.897646 618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:32925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:33.898962 618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:33.900274 618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:33.912004 631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:33.911968 632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:33.912037 635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:33.912820 618 server_base.cc:1034] running on GCE node
I20260809 13:12:33.913650 618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:33.914997 618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:33.916203 618 hybrid_clock.cc:648] HybridClock initialized: now 1786281153916151 us; error 36 us; skew 500 ppm
I20260809 13:12:33.916522 618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:33 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:33.922968 618 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:33.924491 618 webserver.cc:460] Webserver started at http://127.10.181.190:42369/ using document root <none> and password file <none>
I20260809 13:12:33.925176 618 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:33.925295 618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:33.925638 618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:33.928570 618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "fb9f9e2bb4ee411fabea61654aaac7be"
format_stamp: "Formatted at 2026-08-09 13:12:33 on dist-test-slave-42z9"
server_key: "71d0ef68e78315ebdb95b69ccb30aad7"
server_key_iv: "346f2d756e766384e9e558b85e3effd1"
server_key_version: "encryptionkey@0"
I20260809 13:12:33.929314 618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "fb9f9e2bb4ee411fabea61654aaac7be"
format_stamp: "Formatted at 2026-08-09 13:12:33 on dist-test-slave-42z9"
server_key: "71d0ef68e78315ebdb95b69ccb30aad7"
server_key_iv: "346f2d756e766384e9e558b85e3effd1"
server_key_version: "encryptionkey@0"
I20260809 13:12:33.934360 618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:12:33.937656 643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:33.938894 618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:12:33.939055 618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "fb9f9e2bb4ee411fabea61654aaac7be"
format_stamp: "Formatted at 2026-08-09 13:12:33 on dist-test-slave-42z9"
server_key: "71d0ef68e78315ebdb95b69ccb30aad7"
server_key_iv: "346f2d756e766384e9e558b85e3effd1"
server_key_version: "encryptionkey@0"
I20260809 13:12:33.939227 618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:33.951313 618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:33.954883 618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:33.955124 618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:33.972182 618 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:32925
I20260809 13:12:33.972205 726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:32925 every 8 connection(s)
I20260809 13:12:33.973605 618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:33.974259 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 618
I20260809 13:12:33.974473 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:12:33.975214 10966 external_mini_cluster.cc:1428] Setting key 5bfac542cda93fc1f1bf9cb6e11a80fd
I20260809 13:12:33.977839 727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:12:33 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:33.990599 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap starting.
I20260809 13:12:33.994688 727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:33.995904 727 log.cc:824] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:33.998728 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:33.977125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59434 (local address 127.10.181.190:32925)
0809 13:12:33.977919 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:33.977932 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:33.977983 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:12:33.979571 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:33.979594 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:33.979955 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:33.980286 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:33.982086 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:33.983473 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:33.985210 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:33.985970 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:33.986100 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:33.990734 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:33.990746 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:33.990763 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:33.990807 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:33.994442 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:33.995132 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:33.995141 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:33.995149 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:33.995354 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:33.995677 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:33.995682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:33.995685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:33.996352 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:33.996699 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:33.997104 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 13:12:33.997473 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":352,"threads_started":1}
I20260809 13:12:33.999424 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: No bootstrap required, opened a new log
I20260809 13:12:34.006701 727 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:34.007030 727 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:34.007154 727 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Initialized, Role: FOLLOWER
I20260809 13:12:34.007730 727 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:34.007953 727 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:34.008046 727 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:34.008181 727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:34.010231 727 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:34.010653 727 leader_election.cc:302] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb9f9e2bb4ee411fabea61654aaac7be; no voters:
I20260809 13:12:34.011154 727 leader_election.cc:288] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:12:34.011283 735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:34.011896 735 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 LEADER]: Becoming Leader. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Running, Role: LEADER
I20260809 13:12:34.012396 735 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:34.013025 727 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:34.015015 738 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:34.015386 738 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:34.016480 748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:34.016094 739 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb9f9e2bb4ee411fabea61654aaac7be. Latest consensus state: current_term: 1 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:34.017805 739 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:34.022149 748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:34.032322 748 catalog_manager.cc:1347] Generated new cluster ID: 52f1dffeabf14fbc8cb2c2cf0612382b
I20260809 13:12:34.032436 748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:34.042102 748 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:12:34.043849 748 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:34.055358 748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Generated new TSK 0
I20260809 13:12:34.056315 748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:12:34.132794 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:34.470856 766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:34.471138 766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:34.471280 766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:34.478323 766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:34.478441 766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:34.478588 766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:34.487999 766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:34.489238 766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:34.490545 766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:34.502954 772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:34.503033 771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:34.503522 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:34.503973 766 server_base.cc:1034] running on GCE node
I20260809 13:12:34.504518 766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:34.505367 766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:34.506587 766 hybrid_clock.cc:648] HybridClock initialized: now 1786281154506538 us; error 40 us; skew 500 ppm
I20260809 13:12:34.506875 766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:34 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281154, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:34.509862 766 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:34.511196 766 webserver.cc:460] Webserver started at http://127.10.181.129:42613/ using document root <none> and password file <none>
I20260809 13:12:34.511885 766 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:34.512023 766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:34.512316 766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:34.515205 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "3efcbbad864446bc90aadaf6ed2a087c"
format_stamp: "Formatted at 2026-08-09 13:12:34 on dist-test-slave-42z9"
server_key: "c2b59ea444c326fa0211e6e289cdbf30"
server_key_iv: "f94b6f7606c19daf5c4d6e55e08793ec"
server_key_version: "encryptionkey@0"
I20260809 13:12:34.515909 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "3efcbbad864446bc90aadaf6ed2a087c"
format_stamp: "Formatted at 2026-08-09 13:12:34 on dist-test-slave-42z9"
server_key: "c2b59ea444c326fa0211e6e289cdbf30"
server_key_iv: "f94b6f7606c19daf5c4d6e55e08793ec"
server_key_version: "encryptionkey@0"
I20260809 13:12:34.520902 766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:12:34.524211 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:34.525413 766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:34.525604 766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "3efcbbad864446bc90aadaf6ed2a087c"
format_stamp: "Formatted at 2026-08-09 13:12:34 on dist-test-slave-42z9"
server_key: "c2b59ea444c326fa0211e6e289cdbf30"
server_key_iv: "f94b6f7606c19daf5c4d6e55e08793ec"
server_key_version: "encryptionkey@0"
I20260809 13:12:34.525770 766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:34.543637 766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:34.547441 766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:34.547686 766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:34.548561 766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:34.549871 766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:34.549944 766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:34.550040 766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:34.550083 766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:34.580387 766 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:42067
I20260809 13:12:34.580406 967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:42067 every 8 connection(s)
I20260809 13:12:34.581708 766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:34.590852 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 766
I20260809 13:12:34.591078 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:12:34.591867 10966 external_mini_cluster.cc:1428] Setting key e89fb48e6ee90cd0283bccc8a3e7951a
Aug 09 13:12:34 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281154, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:34.601666 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:34.584193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:57123 (local address 127.10.181.190:32925)
0809 13:12:34.584642 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:34.584649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:34.585698 (+ 1049us) server_negotiation.cc:407] Connection header received
0809 13:12:34.587449 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:34.587454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:34.587530 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:34.587629 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:34.589006 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:34.589743 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:34.591717 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:34.592144 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:34.592237 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:34.596199 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:34.596204 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:34.596208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:34.596238 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:34.598657 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:34.599465 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:34.599469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:34.599471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:34.599582 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:34.600012 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:34.600015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:34.600017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:34.600261 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:34.600369 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:34.601179 (+ 810us) server_negotiation.cc:299] Negotiation successful
0809 13:12:34.601348 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":198,"threads_started":1}
I20260809 13:12:34.602999 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:34.584783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:57123)
0809 13:12:34.585552 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:34.585591 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:34.587212 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:34.587816 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:34.587824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:34.588314 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:34.588871 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:34.588889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:34.589895 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:34.589900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:34.591569 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:34.591586 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:34.592373 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:34.592381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:34.592636 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:34.593325 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:34.593353 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:34.595967 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:34.598794 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:34.598802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:34.598818 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:34.599348 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:34.599673 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:34.599677 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:34.599680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:34.599921 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:34.600374 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:34.600383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:34.600800 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 13:12:34.601114 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 13:12:34.601652 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":280,"threads_started":1}
I20260809 13:12:34.604473 969 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:34.604782 969 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:34.605576 969 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:34.607812 669 ts_manager.cc:194] Registered new tserver with Master: 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067)
I20260809 13:12:34.609251 669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:57123
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:12:34.656111 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:34.989988 979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:34.990257 979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:34.990393 979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:34.997377 979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:34.997519 979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:34.997664 979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:35.007077 979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:35.008289 979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:35.009608 979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:35.022316 992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:35.022267 993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:35.022351 996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:35.023386 979 server_base.cc:1034] running on GCE node
I20260809 13:12:35.024017 979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:35.024911 979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:35.026110 979 hybrid_clock.cc:648] HybridClock initialized: now 1786281155026053 us; error 37 us; skew 500 ppm
I20260809 13:12:35.026412 979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:35 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281155, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:35.029458 979 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:35.030882 979 webserver.cc:460] Webserver started at http://127.10.181.130:42841/ using document root <none> and password file <none>
I20260809 13:12:35.031564 979 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:35.031668 979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:35.031921 979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:35.034832 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "0ec4126c1abf498e943da54ea79acc13"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "2f4444c5cc4718d552a2ba6a5cbf63d7"
server_key_iv: "674c7c602c4490c252936dc42a646fee"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.035565 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "0ec4126c1abf498e943da54ea79acc13"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "2f4444c5cc4718d552a2ba6a5cbf63d7"
server_key_iv: "674c7c602c4490c252936dc42a646fee"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.040521 979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:12:35.043993 1006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.045239 979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:35.045392 979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "0ec4126c1abf498e943da54ea79acc13"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "2f4444c5cc4718d552a2ba6a5cbf63d7"
server_key_iv: "674c7c602c4490c252936dc42a646fee"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.045624 979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:35.080489 979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:35.084308 979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:35.084571 979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:35.085558 979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:35.086884 979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:35.086962 979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.087095 979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:35.087149 979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.117116 979 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43231
I20260809 13:12:35.117163 1180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43231 every 8 connection(s)
I20260809 13:12:35.118366 979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:35.122633 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 979
I20260809 13:12:35.122803 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:12:35.123224 10966 external_mini_cluster.cc:1428] Setting key 056e6eefe66d32ff78889040769549fd
Aug 09 13:12:35 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281155, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:35.138504 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.120798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:42993 (local address 127.10.181.190:32925)
0809 13:12:35.121241 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.121246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.122196 (+ 950us) server_negotiation.cc:407] Connection header received
0809 13:12:35.124162 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.124168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.124249 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.124347 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:35.126006 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.126747 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.128234 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.128666 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.128801 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.132697 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:35.132702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:35.132705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:35.132736 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:35.135143 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.135903 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.135907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.135909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.136026 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.136435 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.136438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.136439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.136766 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:35.136916 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.137859 (+ 943us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.138141 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":223,"threads_started":1}
I20260809 13:12:35.139400 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.121261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:42993)
0809 13:12:35.121962 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.122001 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.123860 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.124596 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.124604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.125216 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:35.125841 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.125861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.126888 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.126892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.128097 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.128110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.128919 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.128936 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.129218 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.129915 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:35.129949 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:35.132476 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:35.135277 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:35.135285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:35.135299 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:35.135804 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:35.136121 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:35.136125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:35.136128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:35.136345 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:35.136909 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:35.136917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:35.137330 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.137678 (+ 348us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.138185 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":290,"threads_started":1}
I20260809 13:12:35.141151 1181 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:35.141462 1181 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:35.142316 1181 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:35.143923 669 ts_manager.cc:194] Registered new tserver with Master: 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231)
I20260809 13:12:35.144805 669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:42993
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:12:35.188645 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:35.517163 1192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:35.517418 1192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:35.517552 1192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:35.524552 1192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:35.524662 1192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:35.524780 1192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:12:35.534361 1192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:35.535537 1192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:35.536818 1192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:35.549719 1201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:35.549698 1205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:35.550166 1200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:35.550971 1192 server_base.cc:1034] running on GCE node
I20260809 13:12:35.551539 1192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:35.552414 1192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:35.553603 1192 hybrid_clock.cc:648] HybridClock initialized: now 1786281155553530 us; error 60 us; skew 500 ppm
I20260809 13:12:35.553894 1192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:35 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281155, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:35.556861 1192 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:12:35.558247 1192 webserver.cc:460] Webserver started at http://127.10.181.131:35037/ using document root <none> and password file <none>
I20260809 13:12:35.558905 1192 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:35.559022 1192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:35.559319 1192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:35.562176 1192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "c5ec222c260ba7a1a96a7abb31d63b87"
server_key_iv: "34045551ecdbab37a30593c86a8b120e"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.562953 1192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "c5ec222c260ba7a1a96a7abb31d63b87"
server_key_iv: "34045551ecdbab37a30593c86a8b120e"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.567790 1192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:12:35.571143 1212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.572417 1192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:12:35.572579 1192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "c5ec222c260ba7a1a96a7abb31d63b87"
server_key_iv: "34045551ecdbab37a30593c86a8b120e"
server_key_version: "encryptionkey@0"
I20260809 13:12:35.572763 1192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:35.603454 1192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:35.607381 1192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:35.607653 1192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:35.608530 1192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:35.609807 1192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:35.609885 1192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.610001 1192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:35.610062 1192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:35.612594 969 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:35.641523 1192 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41315
I20260809 13:12:35.641572 1395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41315 every 8 connection(s)
I20260809 13:12:35.642745 1192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:12:35.644258 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 1192
I20260809 13:12:35.644428 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:12:35.644855 10966 external_mini_cluster.cc:1428] Setting key efc608060c218d8b834050911bfc11ad
Aug 09 13:12:35 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281155, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:35.662597 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.646692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:43233 (local address 127.10.181.190:32925)
0809 13:12:35.647237 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.647242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.648206 (+ 964us) server_negotiation.cc:407] Connection header received
0809 13:12:35.650054 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.650058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.650135 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.650223 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:35.651571 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.652357 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.653846 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.654220 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.654309 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.657345 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:35.657350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:35.657353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:35.657384 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:35.660157 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.660773 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.660776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.660778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.660891 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.661214 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.661217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.661219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.661497 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:35.661613 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.662185 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.662340 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":256,"threads_started":1}
I20260809 13:12:35.663448 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.647287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:43233)
0809 13:12:35.648050 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.648104 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.649834 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.650427 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.650435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.650911 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:35.651443 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.651459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.652482 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.652485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.653733 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.653745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.654379 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.654384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.654571 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.655163 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:35.655185 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:35.657158 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:35.660286 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:35.660293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:35.660303 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:35.660688 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:35.660965 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:35.660968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:35.660970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:35.661137 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:35.661609 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:35.661614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:35.661891 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.662099 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.662432 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":323,"threads_started":1}
I20260809 13:12:35.664674 1399 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:35.664940 1399 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:35.665663 1399 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:35.667001 669 ts_manager.cc:194] Registered new tserver with Master: 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:35.667783 669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:43233
I20260809 13:12:35.672084 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:12:35.687595 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.676496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.10.181.190:32925)
0809 13:12:35.676663 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.676669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.677039 (+ 370us) server_negotiation.cc:407] Connection header received
0809 13:12:35.677183 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.677187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.677245 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.677318 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:35.678575 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.679272 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.680522 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.680904 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.680989 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.683205 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:35.683210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:35.683212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:35.683242 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:35.685417 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.686043 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.686046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.686049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.686174 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:35.686447 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:35.686450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:35.686452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:35.686746 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:35.686926 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.687156 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.687313 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 13:12:35.691169 669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:12:35.697839 669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:12:35.724740 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.709333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43231 (local address 127.0.0.1:45892)
0809 13:12:35.715294 (+ 5961us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.715352 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.715539 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.716457 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.716465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.716496 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:35.716980 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.716993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.719041 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.719046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.721435 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.721467 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.723667 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.723673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.723861 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.723911 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.724052 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.724237 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5718,"spinlock_wait_cycles":2816,"thread_start_us":5563,"threads_started":1}
I20260809 13:12:35.725767 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.709617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.10.181.130:43231)
0809 13:12:35.714883 (+ 5266us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.714893 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.715638 (+ 745us) server_negotiation.cc:407] Connection header received
0809 13:12:35.715748 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.715755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.715993 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.716177 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:35.717204 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.718873 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.721675 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.723135 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.723304 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.723510 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.725224 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.725402 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5085,"thread_start_us":4943,"threads_started":1}
I20260809 13:12:35.726722 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.709579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52416)
0809 13:12:35.715332 (+ 5753us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.715358 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.715539 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.717532 (+ 1993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.717537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.717562 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:35.717962 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.717974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.719982 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.719988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.723194 (+ 3206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.723209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.725818 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.725826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.726019 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.726056 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.726150 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.726273 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5637,"spinlock_wait_cycles":8448,"thread_start_us":5489,"threads_started":1}
I20260809 13:12:35.727438 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.710251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.10.181.129:42067)
0809 13:12:35.714405 (+ 4154us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.714415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.715549 (+ 1134us) server_negotiation.cc:407] Connection header received
0809 13:12:35.715777 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.715783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.715990 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.716180 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:35.718173 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.719815 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.723363 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.724788 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.724986 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.725159 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.726945 (+ 1786us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.727124 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3926,"thread_start_us":3698,"threads_started":1}
I20260809 13:12:35.730212 1079 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 (DEFAULT_TABLE table=test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:35.731014 862 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 (DEFAULT_TABLE table=test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:35.731775 1079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4fad9e7abe4d15b9c044188e9d3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:35.732302 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.715905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41315 (local address 127.0.0.1:49216)
0809 13:12:35.724921 (+ 9016us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.724945 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.725060 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.726471 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.726475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.726500 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:35.726923 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.726936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.728737 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.728743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.730208 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.730221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.731633 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.731640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.731797 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.731827 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.731891 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.732007 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8886,"thread_start_us":9240,"threads_started":1}
I20260809 13:12:35.732717 862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4fad9e7abe4d15b9c044188e9d3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:35.733093 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.715924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.10.181.131:41315)
0809 13:12:35.725822 (+ 9898us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.725829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.725864 (+ 35us) server_negotiation.cc:407] Connection header received
0809 13:12:35.725967 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.725974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.726231 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.726511 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:35.727129 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.728606 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.730416 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.731412 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.732568 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.732678 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.732727 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.732861 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9739,"thread_start_us":9581,"threads_started":1}
I20260809 13:12:35.735736 1285 tablet_service.cc:1467] Processing CreateTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 (DEFAULT_TABLE table=test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:35.736781 1285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a4fad9e7abe4d15b9c044188e9d3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:35.747822 1425 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Bootstrap starting.
I20260809 13:12:35.747993 1427 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Bootstrap starting.
I20260809 13:12:35.748595 1428 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Bootstrap starting.
I20260809 13:12:35.751264 1427 tablet_bootstrap.cc:654] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:35.752040 1428 tablet_bootstrap.cc:654] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:35.752226 1427 log.cc:824] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:35.752511 1425 tablet_bootstrap.cc:654] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:35.753288 1428 log.cc:824] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:35.753733 1425 log.cc:824] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:35.756108 1427 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: No bootstrap required, opened a new log
I20260809 13:12:35.756340 1427 ts_tablet_manager.cc:1397] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 13:12:35.756395 1428 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: No bootstrap required, opened a new log
I20260809 13:12:35.756525 1425 tablet_bootstrap.cc:492] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: No bootstrap required, opened a new log
I20260809 13:12:35.756651 1428 ts_tablet_manager.cc:1397] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 13:12:35.756780 1425 ts_tablet_manager.cc:1397] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:12:35.763463 1427 raft_consensus.cc:348] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.763808 1427 raft_consensus.cc:374] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:35.763904 1427 raft_consensus.cc:729] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efcbbad864446bc90aadaf6ed2a087c, State: Initialized, Role: FOLLOWER
I20260809 13:12:35.764449 1427 consensus_queue.cc:260] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.766141 1427 ts_tablet_manager.cc:1428] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260809 13:12:35.766930 1425 raft_consensus.cc:348] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.766930 1428 raft_consensus.cc:348] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.767364 1425 raft_consensus.cc:374] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:35.767366 1428 raft_consensus.cc:374] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:35.767468 1425 raft_consensus.cc:729] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec4126c1abf498e943da54ea79acc13, State: Initialized, Role: FOLLOWER
I20260809 13:12:35.767475 1428 raft_consensus.cc:729] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0ec46d2ced41a68fbae8fe0db0ea86, State: Initialized, Role: FOLLOWER
I20260809 13:12:35.768047 1428 consensus_queue.cc:260] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.768047 1425 consensus_queue.cc:260] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.769598 1181 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:35.769658 1399 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:35.769829 1428 ts_tablet_manager.cc:1428] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 13:12:35.772818 1425 ts_tablet_manager.cc:1428] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260809 13:12:35.833892 970 tablet.cc:2369] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:12:35.870333 1183 tablet.cc:2369] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:12:35.894351 1400 tablet.cc:2369] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:35.948175 1433 raft_consensus.cc:482] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:12:35.948459 1433 raft_consensus.cc:504] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.950306 1433 leader_election.cc:288] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231), 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:35.957912 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.950747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:35133 (local address 127.10.181.130:43231)
0809 13:12:35.950940 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.950946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.951491 (+ 545us) server_negotiation.cc:407] Connection header received
0809 13:12:35.951718 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.951724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.951806 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.951904 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:35.952690 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.953676 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.955979 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.957077 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.957229 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.957304 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.957570 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.957710 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:12:35.958060 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.950676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43231 (local address 127.10.181.129:35133)
0809 13:12:35.951390 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.951408 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.951554 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.951958 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.951963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.951991 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:35.952506 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.952519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.953795 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.953799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.955820 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.955836 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.957270 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.957274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.957392 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.957417 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.957526 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.957652 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":3,"thread_start_us":398,"threads_started":1}
I20260809 13:12:35.959013 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.951722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:37535 (local address 127.10.181.131:41315)
0809 13:12:35.951959 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:35.951966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:35.952636 (+ 670us) server_negotiation.cc:407] Connection header received
0809 13:12:35.952801 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:35.952806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:35.952877 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:35.952969 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:35.953375 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.954204 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.956351 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.957496 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.957647 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.957712 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:35.958665 (+ 953us) server_negotiation.cc:299] Negotiation successful
0809 13:12:35.958782 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:12:35.959043 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:35.951547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41315 (local address 127.10.181.129:37535)
0809 13:12:35.952547 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:35.952562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:35.952684 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:35.952971 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:35.952974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:35.952990 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:35.953252 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.953260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.954340 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.954346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.956219 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:35.956231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:35.958341 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:35.958350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:35.958485 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:35.958509 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:12:35.958597 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 13:12:35.958713 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":608,"threads_started":1}
I20260809 13:12:35.959185 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4fad9e7abe4d15b9c044188e9d3273" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec4126c1abf498e943da54ea79acc13" is_pre_election: true
I20260809 13:12:35.959810 1109 raft_consensus.cc:2454] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3efcbbad864446bc90aadaf6ed2a087c in term 0.
I20260809 13:12:35.960511 1320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4fad9e7abe4d15b9c044188e9d3273" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" is_pre_election: true
I20260809 13:12:35.961057 1320 raft_consensus.cc:2454] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3efcbbad864446bc90aadaf6ed2a087c in term 0.
I20260809 13:12:35.967304 792 leader_election.cc:302] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3efcbbad864446bc90aadaf6ed2a087c, 9a0ec46d2ced41a68fbae8fe0db0ea86; no voters:
I20260809 13:12:35.967670 1433 raft_consensus.cc:2781] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:12:35.967837 1433 raft_consensus.cc:482] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:12:35.967959 1433 raft_consensus.cc:3037] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:35.972455 1433 raft_consensus.cc:504] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.973312 1433 leader_election.cc:288] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 election: Requested vote from peers 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231), 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:35.973821 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4fad9e7abe4d15b9c044188e9d3273" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec4126c1abf498e943da54ea79acc13"
I20260809 13:12:35.974040 1109 raft_consensus.cc:3037] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:35.974117 1320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a4fad9e7abe4d15b9c044188e9d3273" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
I20260809 13:12:35.974311 1320 raft_consensus.cc:3037] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:35.977370 1109 raft_consensus.cc:2454] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3efcbbad864446bc90aadaf6ed2a087c in term 1.
I20260809 13:12:35.977883 792 leader_election.cc:302] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec4126c1abf498e943da54ea79acc13, 3efcbbad864446bc90aadaf6ed2a087c; no voters:
I20260809 13:12:35.978133 1320 raft_consensus.cc:2454] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3efcbbad864446bc90aadaf6ed2a087c in term 1.
I20260809 13:12:35.978173 1433 raft_consensus.cc:2781] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:35.985739 1433 raft_consensus.cc:686] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 1 LEADER]: Becoming Leader. State: Replica: 3efcbbad864446bc90aadaf6ed2a087c, State: Running, Role: LEADER
I20260809 13:12:35.986387 1433 consensus_queue.cc:237] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:35.991642 669 catalog_manager.cc:5495] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129). New cstate: current_term: 1 leader_uuid: "3efcbbad864446bc90aadaf6ed2a087c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } health_report { overall_health: UNKNOWN } } }
I20260809 13:12:36.012555 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:36.006387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.10.181.129:42067)
0809 13:12:36.006568 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:36.006574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:36.006748 (+ 174us) server_negotiation.cc:407] Connection header received
0809 13:12:36.006936 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:36.006943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:36.007047 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:36.007204 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:12:36.007497 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.008614 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.010046 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.010741 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.010871 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:36.011295 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:12:36.011980 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:12:36.012135 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:36.012184 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 13:12:36.012272 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 13:12:36.020967 1109 raft_consensus.cc:1261] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Refusing update from remote peer 3efcbbad864446bc90aadaf6ed2a087c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:36.021162 1320 raft_consensus.cc:1261] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Refusing update from remote peer 3efcbbad864446bc90aadaf6ed2a087c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:36.021942 1433 consensus_queue.cc:1035] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:36.022315 1441 consensus_queue.cc:1035] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:36.038130 1445 mvcc.cc:205] Tried to move back new op lower bound from 7316607615051632640 to 7316607614925025280. Current Snapshot: MvccSnapshot[applied={T|T < 7316607615051632640}]
I20260809 13:12:36.040133 1446 mvcc.cc:205] Tried to move back new op lower bound from 7316607615051632640 to 7316607614925025280. Current Snapshot: MvccSnapshot[applied={T|T < 7316607615051632640}]
I20260809 13:12:36.042379 1447 mvcc.cc:205] Tried to move back new op lower bound from 7316607615051632640 to 7316607614925025280. Current Snapshot: MvccSnapshot[applied={T|T < 7316607615051632640}]
I20260809 13:12:36.062310 669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:36.062613 669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:36.066602 669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Sending DeleteTablet for 3 replicas of tablet 6a4fad9e7abe4d15b9c044188e9d3273
I20260809 13:12:36.067828 1079 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:36 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:45892
I20260809 13:12:36.067883 1285 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:36 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49216
I20260809 13:12:36.068071 862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a4fad9e7abe4d15b9c044188e9d3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:36 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:52416
I20260809 13:12:36.069038 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 766
I20260809 13:12:36.079172 1459 tablet_replica.cc:330] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: stopping tablet replica
I20260809 13:12:36.079813 1456 tablet_replica.cc:330] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: stopping tablet replica
I20260809 13:12:36.081543 1456 raft_consensus.cc:2229] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:36.082515 1459 raft_consensus.cc:2229] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:12:36.085709 1459 raft_consensus.cc:2258] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:36.086320 1456 raft_consensus.cc:2258] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:36.086895 1458 tablet_replica.cc:330] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: stopping tablet replica
I20260809 13:12:36.088002 1458 raft_consensus.cc:2229] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:36.088482 1458 raft_consensus.cc:2258] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:36.089618 1456 ts_tablet_manager.cc:1905] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:36.090557 1459 ts_tablet_manager.cc:1905] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:36.094980 1458 ts_tablet_manager.cc:1905] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:36.096616 1456 ts_tablet_manager.cc:1918] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:36.096823 1456 log.cc:1196] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/6a4fad9e7abe4d15b9c044188e9d3273
Aug 09 13:12:36 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:12:36.097472 1456 ts_tablet_manager.cc:1939] T 6a4fad9e7abe4d15b9c044188e9d3273 P 0ec4126c1abf498e943da54ea79acc13: Deleting consensus metadata
I20260809 13:12:36.099411 650 catalog_manager.cc:4841] TS 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231): tablet 6a4fad9e7abe4d15b9c044188e9d3273 (table test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]) successfully deleted
I20260809 13:12:36.104095 1458 ts_tablet_manager.cc:1918] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:36.104333 1458 log.cc:1196] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/6a4fad9e7abe4d15b9c044188e9d3273
I20260809 13:12:36.104882 1458 ts_tablet_manager.cc:1939] T 6a4fad9e7abe4d15b9c044188e9d3273 P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting consensus metadata
I20260809 13:12:36.106359 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:36.070517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.10.181.129:42067)
0809 13:12:36.070702 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:36.070710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:36.080515 (+ 9805us) server_negotiation.cc:407] Connection header received
0809 13:12:36.082832 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:36.082838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:36.082947 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:36.083031 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:36.087368 (+ 4337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.089885 (+ 2517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.091867 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.092593 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.092695 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:36.098188 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:36.098204 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:36.098224 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:36.098267 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:36.102323 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.103619 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.103628 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.103632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.103822 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.104298 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.104304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.104307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.104746 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:36.105038 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:36.105696 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 13:12:36.105973 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:12:36.106665 650 catalog_manager.cc:4841] TS 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315): tablet 6a4fad9e7abe4d15b9c044188e9d3273 (table test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]) successfully deleted
I20260809 13:12:36.107985 960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:36.202822 1459 ts_tablet_manager.cc:1918] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:36.203166 1459 log.cc:1196] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/6a4fad9e7abe4d15b9c044188e9d3273
I20260809 13:12:36.203997 1459 ts_tablet_manager.cc:1939] T 6a4fad9e7abe4d15b9c044188e9d3273 P 3efcbbad864446bc90aadaf6ed2a087c: Deleting consensus metadata
I20260809 13:12:36.206164 651 catalog_manager.cc:4841] TS 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067): tablet 6a4fad9e7abe4d15b9c044188e9d3273 (table test-table [id=cb8dd17c0f2c4fb78d6de5d294eb0c27]) successfully deleted
I20260809 13:12:36.213065 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 766
I20260809 13:12:36.240376 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 979
Aug 09 13:12:36 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:12:36.254729 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:36.241201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.10.181.130:43231)
0809 13:12:36.241368 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:36.241372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:36.241391 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:12:36.241474 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:36.241479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:36.241561 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:36.241651 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:36.242942 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.243737 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.245013 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.245369 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.245570 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:36.249495 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:36.249509 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:36.249527 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:36.249565 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:36.252289 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.252943 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.252949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.252952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.253062 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.253343 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.253346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.253348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.253795 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:36.254011 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:36.254254 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 13:12:36.254411 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:12:36.255363 1173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:36.344210 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 979
I20260809 13:12:36.368377 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 1192
Aug 09 13:12:36 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:12:36.382990 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:36.369216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.10.181.131:41315)
0809 13:12:36.369406 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:36.369412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:36.369434 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:12:36.369628 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:36.369633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:36.369705 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:36.369792 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:36.371158 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.371871 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.373156 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:36.373662 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:36.373777 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:36.378092 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:36.378109 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:36.378131 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:36.378177 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:36.380679 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.381269 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.381276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.381278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.381424 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:36.381748 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:36.381751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:36.381753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:36.382104 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:36.382326 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:36.382546 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 13:12:36.382717 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:12:36.383625 1388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:36.470649 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 1192
I20260809 13:12:36.497236 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 618
I20260809 13:12:36.498467 719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:36.607824 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 618
I20260809 13:12:36.635849 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--webserver_interface=127.10.181.190
--webserver_port=42369
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:36.960831 1466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:36.961095 1466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:36.961227 1466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:36.968323 1466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:36.968433 1466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:36.968513 1466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:36.968583 1466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:36.968643 1466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:36.978647 1466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:32925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=42369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:36.979982 1466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:36.981343 1466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:37.005913 1478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.005993 1474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.005968 1475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:37.006873 1466 server_base.cc:1034] running on GCE node
I20260809 13:12:37.007741 1466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:37.009254 1466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:37.010501 1466 hybrid_clock.cc:648] HybridClock initialized: now 1786281157010437 us; error 41 us; skew 500 ppm
I20260809 13:12:37.010922 1466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:37 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:37.014338 1466 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:37.015971 1466 webserver.cc:460] Webserver started at http://127.10.181.190:42369/ using document root <none> and password file <none>
I20260809 13:12:37.016708 1466 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:37.016848 1466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:37.022732 1466 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 13:12:37.025604 1487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:37.026909 1466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:12:37.027084 1466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "fb9f9e2bb4ee411fabea61654aaac7be"
format_stamp: "Formatted at 2026-08-09 13:12:33 on dist-test-slave-42z9"
server_key: "71d0ef68e78315ebdb95b69ccb30aad7"
server_key_iv: "346f2d756e766384e9e558b85e3effd1"
server_key_version: "encryptionkey@0"
I20260809 13:12:37.028110 1466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:37.059159 1466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:37.063606 1466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:37.063876 1466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:37.081936 1466 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:32925
I20260809 13:12:37.081969 1574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:32925 every 8 connection(s)
I20260809 13:12:37.083429 1466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:37.090607 1575 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:12:37.090770 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 1466
I20260809 13:12:37.091773 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:42067
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=42613
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:37.095151 1575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap starting.
I20260809 13:12:37.117264 1575 log.cc:824] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:37.129702 1575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:12:37.130193 1575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap complete.
I20260809 13:12:37.139453 1575 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:37.140255 1575 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Initialized, Role: FOLLOWER
I20260809 13:12:37.140856 1575 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:37.141079 1575 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:37.141201 1575 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:37.141359 1575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:12:37.145036 1575 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:37.145605 1575 leader_election.cc:302] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb9f9e2bb4ee411fabea61654aaac7be; no voters:
I20260809 13:12:37.146224 1575 leader_election.cc:288] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 13:12:37.146374 1579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:12:37.147552 1579 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 LEADER]: Becoming Leader. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Running, Role: LEADER
I20260809 13:12:37.148176 1579 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:37.148483 1575 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:37.150605 1582 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb9f9e2bb4ee411fabea61654aaac7be. Latest consensus state: current_term: 2 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:37.151011 1582 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:37.151557 1580 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:37.151818 1580 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:37.153931 1588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:37.164379 1588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:37.166000 1588 catalog_manager.cc:1259] Loaded cluster ID: 52f1dffeabf14fbc8cb2c2cf0612382b
I20260809 13:12:37.166152 1588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:37.169790 1588 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 13:12:37.188889 1588 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 13:12:37.210872 1588 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:37.212839 1588 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Loaded TSK: 0
I20260809 13:12:37.214540 1588 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 13:12:37.460585 1577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:37.460850 1577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:37.460992 1577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:37.468096 1577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:37.468212 1577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:37.468357 1577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:37.477960 1577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:42067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=42613
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:37.479167 1577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:37.480469 1577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:37.492937 1613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.493042 1621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.493227 1615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:37.494563 1577 server_base.cc:1034] running on GCE node
I20260809 13:12:37.495136 1577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:37.496078 1577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:37.497316 1577 hybrid_clock.cc:648] HybridClock initialized: now 1786281157497251 us; error 44 us; skew 500 ppm
I20260809 13:12:37.497651 1577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:37 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:37.500943 1577 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:37.502441 1577 webserver.cc:460] Webserver started at http://127.10.181.129:42613/ using document root <none> and password file <none>
I20260809 13:12:37.503103 1577 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:37.503237 1577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:37.508420 1577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 13:12:37.511138 1633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:37.512420 1577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:12:37.512563 1577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "3efcbbad864446bc90aadaf6ed2a087c"
format_stamp: "Formatted at 2026-08-09 13:12:34 on dist-test-slave-42z9"
server_key: "c2b59ea444c326fa0211e6e289cdbf30"
server_key_iv: "f94b6f7606c19daf5c4d6e55e08793ec"
server_key_version: "encryptionkey@0"
I20260809 13:12:37.513607 1577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:37.529340 1577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:37.533310 1577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:37.533603 1577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:37.534528 1577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:37.535778 1577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:37.535873 1577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:37.535974 1577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:37.536048 1577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:37.566709 1577 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:42067
I20260809 13:12:37.566776 1805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:42067 every 8 connection(s)
I20260809 13:12:37.568138 1577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:37.572259 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 1577
I20260809 13:12:37.572901 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:43231
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=42841
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:37 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:37.591399 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:37.570779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:41765 (local address 127.10.181.190:32925)
0809 13:12:37.571414 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:37.571430 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:37.571898 (+ 468us) server_negotiation.cc:407] Connection header received
0809 13:12:37.574139 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:37.574151 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:37.574564 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:37.574904 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:37.576723 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:37.578000 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:37.580056 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:37.580692 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:37.580815 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:37.584477 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:37.584484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:37.584497 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:37.584537 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:37.587235 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:37.588044 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:37.588050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:37.588056 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:37.588220 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:37.588707 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:37.588710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:37.588712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:37.589128 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:37.589280 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:37.590221 (+ 941us) server_negotiation.cc:299] Negotiation successful
0809 13:12:37.590562 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":245,"threads_started":1}
I20260809 13:12:37.591631 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:37.571006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:41765)
0809 13:12:37.571739 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:37.571774 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:37.573642 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:37.575105 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:37.575115 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:37.575774 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:37.576487 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:37.576512 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:37.578158 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:37.578163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:37.579935 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:37.579951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:37.580930 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:37.580937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:37.581203 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:37.581869 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:37.581904 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:37.584236 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:37.587377 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:37.587385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:37.587399 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:37.587936 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:37.588341 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:37.588347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:37.588351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:37.588614 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:37.589266 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:37.589273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:37.589716 (+ 443us) client_negotiation.cc:769] Sending connection context
0809 13:12:37.590019 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 13:12:37.590497 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":293,"threads_started":1}
I20260809 13:12:37.594564 1808 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:37.594827 1808 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:37.595515 1808 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:37.597883 1521 ts_manager.cc:194] Registered new tserver with Master: 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067)
I20260809 13:12:37.599985 1521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:41765
W20260809 13:12:37.906556 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:37.906792 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:37.906888 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:37.913697 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:37.913789 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:37.913902 1815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:37.922873 1815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:43231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=42841
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:37.923944 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:37.925109 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:37.937400 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.937526 1822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:37.937498 1827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:37.938328 1815 server_base.cc:1034] running on GCE node
I20260809 13:12:37.938856 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:37.939713 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:37.940915 1815 hybrid_clock.cc:648] HybridClock initialized: now 1786281157940866 us; error 35 us; skew 500 ppm
I20260809 13:12:37.941206 1815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:37 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:37.944165 1815 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:37.945418 1815 webserver.cc:460] Webserver started at http://127.10.181.130:42841/ using document root <none> and password file <none>
I20260809 13:12:37.946084 1815 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:37.946215 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:37.951296 1815 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:12:37.953840 1838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:37.955034 1815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:12:37.955173 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "0ec4126c1abf498e943da54ea79acc13"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "2f4444c5cc4718d552a2ba6a5cbf63d7"
server_key_iv: "674c7c602c4490c252936dc42a646fee"
server_key_version: "encryptionkey@0"
I20260809 13:12:37.956056 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:37.993598 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:37.997681 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:37.997928 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:37.998904 1815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:38.000198 1815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:38.000290 1815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:38.000401 1815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:38.000473 1815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:38.030607 1815 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43231
I20260809 13:12:38.030676 2024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43231 every 8 connection(s)
I20260809 13:12:38.032071 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:38.033372 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 1815
I20260809 13:12:38.034052 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:41315
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=35037
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:12:38 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:38.051707 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.034705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:57839 (local address 127.10.181.190:32925)
0809 13:12:38.034870 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.034874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.036293 (+ 1419us) server_negotiation.cc:407] Connection header received
0809 13:12:38.038187 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.038193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.038285 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.038375 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:38.039916 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.040629 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.042273 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.042698 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.042811 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.046217 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:38.046222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:38.046225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:38.046258 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:38.048645 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.049377 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.049381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.049383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.049552 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.050027 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.050032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.050036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.050300 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:38.050435 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.051117 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.051352 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:12:38.052598 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.035362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:57839)
0809 13:12:38.036146 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.036183 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.037907 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.038639 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.038647 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.039145 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:38.039736 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.039754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.040792 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.040796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.042121 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.042135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.042919 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.042926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.043152 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.043776 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:38.043800 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:38.046019 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:38.048782 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:38.048791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:38.048806 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:38.049234 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:38.049677 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:38.049683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:38.049687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:38.049912 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:38.050429 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:38.050436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:38.050753 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.050977 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.051486 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":322,"threads_started":1}
I20260809 13:12:38.053997 2025 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:38.054291 2025 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:38.055014 2025 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:38.056479 1521 ts_manager.cc:194] Registered new tserver with Master: 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231)
I20260809 13:12:38.057653 1521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:57839
W20260809 13:12:38.362515 2029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:38.362756 2029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:38.362854 2029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:38.369675 2029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:38.369766 2029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:38.369880 2029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:12:38.379107 2029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:41315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=35037
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:38.380231 2029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:38.381428 2029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:38.393503 2035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:38.393615 2036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:38.394078 2041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:38.394536 2029 server_base.cc:1034] running on GCE node
I20260809 13:12:38.395107 2029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:38.395965 2029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:38.397183 2029 hybrid_clock.cc:648] HybridClock initialized: now 1786281158397136 us; error 27 us; skew 500 ppm
I20260809 13:12:38.397514 2029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:38 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281158, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:38.400641 2029 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:12:38.401979 2029 webserver.cc:460] Webserver started at http://127.10.181.131:35037/ using document root <none> and password file <none>
I20260809 13:12:38.402662 2029 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:38.402828 2029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:38.407841 2029 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:12:38.410395 2055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:38.411659 2029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:12:38.411803 2029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "c5ec222c260ba7a1a96a7abb31d63b87"
server_key_iv: "34045551ecdbab37a30593c86a8b120e"
server_key_version: "encryptionkey@0"
I20260809 13:12:38.412715 2029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:38.429096 2029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:38.432760 2029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:38.432996 2029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:38.433915 2029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:38.435120 2029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:38.435212 2029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:38.435318 2029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:38.435388 2029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:38.464421 2029 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41315
I20260809 13:12:38.464481 2222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41315 every 8 connection(s)
I20260809 13:12:38.465883 2029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:12:38.471920 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 2029
Aug 09 13:12:38 dist-test-slave-42z9 krb5kdc[596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281158, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:38.485766 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.468303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:55957 (local address 127.10.181.190:32925)
0809 13:12:38.468509 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.468516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.469684 (+ 1168us) server_negotiation.cc:407] Connection header received
0809 13:12:38.471419 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.471424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.471494 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.471581 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:38.473523 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.474505 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.476019 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.476595 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.476706 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.480134 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:38.480139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:38.480142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:38.480171 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:38.482916 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.483581 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.483585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.483587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.483720 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.484132 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.484136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.484138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.484387 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:38.484513 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.485169 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.485400 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":65}
I20260809 13:12:38.486267 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.472932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.10.181.190:32925)
0809 13:12:38.473378 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.473383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.473850 (+ 467us) server_negotiation.cc:407] Connection header received
0809 13:12:38.473993 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.473999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.474078 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.474178 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:38.475711 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.476542 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.477909 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.478350 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.478466 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.481083 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:38.481088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:38.481091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:38.481135 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:38.483673 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.484290 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.484295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.484297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.484414 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.484769 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.484775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.484777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.485022 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:38.485262 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.485716 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.485921 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":335,"thread_start_us":245,"threads_started":1}
I20260809 13:12:38.486765 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.468830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:55957)
0809 13:12:38.469539 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.469585 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.471180 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.471860 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.471871 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.472569 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:38.473260 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.473286 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.474680 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.474684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.475877 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.475890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.476817 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.476825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.477075 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.477699 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:38.477722 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:38.479916 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:38.483041 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:38.483049 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:38.483064 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:38.483454 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:38.483822 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:38.483828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:38.483832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:38.484013 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:38.484506 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:38.484514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:38.484812 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.485035 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.485533 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":267,"threads_started":1}
I20260809 13:12:38.488157 2223 heartbeater.cc:347] Connected to a master server at 127.10.181.190:32925
I20260809 13:12:38.488473 2223 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:38.489204 2223 heartbeater.cc:511] Master 127.10.181.190:32925 requested a full tablet report, sending...
I20260809 13:12:38.490649 1520 ts_manager.cc:194] Registered new tserver with Master: 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:38.491474 1520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:55957
I20260809 13:12:38.500520 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:12:38.506433 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.501343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.10.181.190:32925)
0809 13:12:38.501511 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.501515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.501948 (+ 433us) server_negotiation.cc:407] Connection header received
0809 13:12:38.502107 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.502110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.502236 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.502343 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:12:38.502651 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.503283 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.504665 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.505072 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.505154 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.505635 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:12:38.506036 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:12:38.506142 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.506201 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.506255 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:12:38.507241 1520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:12:38.514269 1520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:12:38.549374 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.525516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43231 (local address 127.0.0.1:45900)
0809 13:12:38.526224 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.526281 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.526439 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.541890 (+ 15451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.541897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.541931 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.542443 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.542456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.544294 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.544300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.546808 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.546823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.548348 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.548354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.548582 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.548644 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.548757 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.548938 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":58,"thread_start_us":310,"threads_started":1}
I20260809 13:12:38.550158 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.526108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.10.181.130:43231)
0809 13:12:38.541210 (+ 15102us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.541217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.541254 (+ 37us) server_negotiation.cc:407] Connection header received
0809 13:12:38.541358 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.541366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.541653 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.541858 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.542638 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.544142 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.546964 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.548166 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.548340 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.549558 (+ 1218us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.549624 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.549824 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14920,"thread_start_us":15989,"threads_started":1}
I20260809 13:12:38.550344 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.526350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52428)
0809 13:12:38.541359 (+ 15009us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.541382 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.541511 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.543802 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.543806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.543850 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.544203 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.544216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.545765 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.545770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.547561 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.547585 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.549482 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.549489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.549718 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.549780 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.549858 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.549978 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14891,"mutex_wait_us":34,"thread_start_us":15531,"threads_started":1}
I20260809 13:12:38.550464 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.527134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.10.181.131:41315)
0809 13:12:38.537147 (+ 10013us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.537156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.542317 (+ 5161us) server_negotiation.cc:407] Connection header received
0809 13:12:38.542460 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.542466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.542633 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.542810 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.543591 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.545074 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.547460 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.548974 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.549168 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.549348 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.550009 (+ 661us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.550180 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9821,"thread_start_us":9647,"threads_started":1}
I20260809 13:12:38.551056 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.526653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41315 (local address 127.0.0.1:49242)
0809 13:12:38.542178 (+ 15525us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.542194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.542317 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.543085 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.543090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.543114 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.543396 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.543407 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.545255 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.545260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.547316 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.547327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.549518 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.549524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.549700 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.549776 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.549858 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.550695 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15392,"mutex_wait_us":48,"thread_start_us":14937,"threads_started":1}
I20260809 13:12:38.551395 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.526898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.10.181.129:42067)
0809 13:12:38.543332 (+ 16434us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.543337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.543358 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:12:38.543426 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.543431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.543607 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.543810 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.544346 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.545616 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.547752 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.549184 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.550695 (+ 1511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.550871 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.550938 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.551095 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16289,"thread_start_us":16150,"threads_started":1}
I20260809 13:12:38.554145 1922 tablet_service.cc:1467] Processing CreateTablet for tablet 5733cd5adb1d44228b27c397d386b7bb (DEFAULT_TABLE table=test-table [id=47e46c9735bd42989a43833bcb49d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:38.554684 2127 tablet_service.cc:1467] Processing CreateTablet for tablet 5733cd5adb1d44228b27c397d386b7bb (DEFAULT_TABLE table=test-table [id=47e46c9735bd42989a43833bcb49d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:38.554847 1705 tablet_service.cc:1467] Processing CreateTablet for tablet 5733cd5adb1d44228b27c397d386b7bb (DEFAULT_TABLE table=test-table [id=47e46c9735bd42989a43833bcb49d919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:12:38.555645 1922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5733cd5adb1d44228b27c397d386b7bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:38.555823 2127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5733cd5adb1d44228b27c397d386b7bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:38.556129 1705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5733cd5adb1d44228b27c397d386b7bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:12:38.575834 2248 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Bootstrap starting.
I20260809 13:12:38.576812 2249 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Bootstrap starting.
I20260809 13:12:38.580333 2248 tablet_bootstrap.cc:654] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:38.580395 2247 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Bootstrap starting.
I20260809 13:12:38.581298 2249 tablet_bootstrap.cc:654] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:38.581702 2248 log.cc:824] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:38.582546 2249 log.cc:824] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:38.583235 2247 tablet_bootstrap.cc:654] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:38.584151 2247 log.cc:824] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:38.584955 2248 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: No bootstrap required, opened a new log
I20260809 13:12:38.585214 2248 ts_tablet_manager.cc:1397] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260809 13:12:38.586730 2249 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: No bootstrap required, opened a new log
I20260809 13:12:38.586978 2249 ts_tablet_manager.cc:1397] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 13:12:38.587070 2247 tablet_bootstrap.cc:492] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: No bootstrap required, opened a new log
I20260809 13:12:38.587318 2247 ts_tablet_manager.cc:1397] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 13:12:38.594367 2249 raft_consensus.cc:348] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.594718 2249 raft_consensus.cc:374] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:38.594817 2249 raft_consensus.cc:729] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efcbbad864446bc90aadaf6ed2a087c, State: Initialized, Role: FOLLOWER
I20260809 13:12:38.595368 2249 consensus_queue.cc:260] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.595302 2248 raft_consensus.cc:348] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.595728 2248 raft_consensus.cc:374] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:38.595862 2248 raft_consensus.cc:729] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec4126c1abf498e943da54ea79acc13, State: Initialized, Role: FOLLOWER
I20260809 13:12:38.596393 1808 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:38.596556 2248 consensus_queue.cc:260] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.597019 2249 ts_tablet_manager.cc:1428] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 13:12:38.597267 2247 raft_consensus.cc:348] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.597690 2247 raft_consensus.cc:374] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:38.597841 2247 raft_consensus.cc:729] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0ec46d2ced41a68fbae8fe0db0ea86, State: Initialized, Role: FOLLOWER
I20260809 13:12:38.598415 2247 consensus_queue.cc:260] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.599579 2223 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:38.600283 2247 ts_tablet_manager.cc:1428] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260809 13:12:38.602738 2025 heartbeater.cc:503] Master 127.10.181.190:32925 was elected leader, sending a full tablet report...
I20260809 13:12:38.603153 2248 ts_tablet_manager.cc:1428] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Time spent starting tablet: real 0.018s user 0.007s sys 0.010s
I20260809 13:12:38.646695 2256 raft_consensus.cc:482] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:12:38.646979 2256 raft_consensus.cc:504] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.648960 2256 leader_election.cc:288] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067), 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:38.658115 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.649777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59621 (local address 127.10.181.129:42067)
0809 13:12:38.650033 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.650097 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.650745 (+ 648us) server_negotiation.cc:407] Connection header received
0809 13:12:38.650947 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.650951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.651026 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.651128 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.651763 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.652845 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.655797 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.656987 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.657137 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.657213 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.657735 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.657866 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 13:12:38.658130 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.649327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51103 (local address 127.10.181.131:41315)
0809 13:12:38.649647 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.649654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.650833 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 13:12:38.650946 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.650951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.651026 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.651128 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.651849 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.652876 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.655800 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.656983 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.657141 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.657213 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.657735 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.657866 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260809 13:12:38.658331 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.649322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.10.181.130:59621)
0809 13:12:38.650638 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.650658 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.650810 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.651143 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.651148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.651175 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.651612 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.651636 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.652968 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.652973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.655650 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.655666 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.657238 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.657243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.657559 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.657610 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.657707 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.657916 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"mutex_wait_us":242,"thread_start_us":984,"threads_started":1}
I20260809 13:12:38.658699 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.649693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41315 (local address 127.10.181.130:51103)
0809 13:12:38.650739 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.650757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.650852 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.651154 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.651158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.651243 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.651706 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.651717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.652973 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.652977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.655649 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.655666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.657232 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.657239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.657554 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.657610 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.657707 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.657916 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":231,"spinlock_wait_cycles":22272,"thread_start_us":505,"threads_started":1}
I20260809 13:12:38.658742 2253 raft_consensus.cc:482] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:12:38.659044 2253 raft_consensus.cc:504] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.659551 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "0ec4126c1abf498e943da54ea79acc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efcbbad864446bc90aadaf6ed2a087c" is_pre_election: true
I20260809 13:12:38.659736 2153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "0ec4126c1abf498e943da54ea79acc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" is_pre_election: true
I20260809 13:12:38.660027 1736 raft_consensus.cc:2454] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec4126c1abf498e943da54ea79acc13 in term 0.
I20260809 13:12:38.660282 2153 raft_consensus.cc:2454] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec4126c1abf498e943da54ea79acc13 in term 0.
I20260809 13:12:38.660771 1848 leader_election.cc:302] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec4126c1abf498e943da54ea79acc13, 3efcbbad864446bc90aadaf6ed2a087c; no voters:
I20260809 13:12:38.660946 2253 leader_election.cc:288] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231), 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:38.661181 2256 raft_consensus.cc:2781] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:12:38.661329 2256 raft_consensus.cc:482] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:12:38.661430 2256 raft_consensus.cc:3037] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:38.664688 2256 raft_consensus.cc:504] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.665885 2153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "0ec4126c1abf498e943da54ea79acc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
I20260809 13:12:38.666074 2153 raft_consensus.cc:3037] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:38.668426 2153 raft_consensus.cc:2454] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec4126c1abf498e943da54ea79acc13 in term 1.
I20260809 13:12:38.668946 1847 leader_election.cc:302] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec4126c1abf498e943da54ea79acc13, 9a0ec46d2ced41a68fbae8fe0db0ea86; no voters:
I20260809 13:12:38.669118 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.661129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:43231 (local address 127.10.181.129:33151)
0809 13:12:38.662055 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.662083 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.662247 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.662704 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.662709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.662734 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.663098 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.663109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.664495 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.664499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.666601 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.666618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.668311 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.668318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.668489 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.668522 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.668619 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.668745 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":612,"threads_started":1}
I20260809 13:12:38.669112 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.661249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:33151 (local address 127.10.181.130:43231)
0809 13:12:38.661476 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.661483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.662155 (+ 672us) server_negotiation.cc:407] Connection header received
0809 13:12:38.662379 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.662384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.662449 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.662513 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.663259 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.664321 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.666763 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.668078 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.668241 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.668318 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.668666 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.668818 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:12:38.669879 2256 leader_election.cc:288] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [CANDIDATE]: Term 1 election: Requested vote from peers 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067), 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315)
I20260809 13:12:38.669962 2265 raft_consensus.cc:2781] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:38.670174 2265 raft_consensus.cc:686] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec4126c1abf498e943da54ea79acc13, State: Running, Role: LEADER
I20260809 13:12:38.670765 2265 consensus_queue.cc:237] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } }
I20260809 13:12:38.672910 1736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "0ec4126c1abf498e943da54ea79acc13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efcbbad864446bc90aadaf6ed2a087c"
I20260809 13:12:38.673134 1736 raft_consensus.cc:3037] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:38.675618 1520 catalog_manager.cc:5495] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "0ec4126c1abf498e943da54ea79acc13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec4126c1abf498e943da54ea79acc13" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 } health_report { overall_health: UNKNOWN } } }
I20260809 13:12:38.676187 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.662713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:39253 (local address 127.10.181.131:41315)
0809 13:12:38.662872 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.662877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.669498 (+ 6621us) server_negotiation.cc:407] Connection header received
0809 13:12:38.669666 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.669672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.669765 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.669858 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:12:38.670384 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.671520 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.673842 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.675037 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.675164 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.675234 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.675795 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.675923 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:12:38.676201 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.662173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41315 (local address 127.10.181.129:39253)
0809 13:12:38.669389 (+ 7216us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.669405 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:38.669537 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:38.670028 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:38.670034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:38.670064 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:12:38.670227 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.670237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.671725 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.671730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.673656 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:38.673673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.675414 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.675420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:38.675607 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.675649 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 13:12:38.675735 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 13:12:38.675866 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7116,"thread_start_us":10232,"threads_started":1}
I20260809 13:12:38.676344 1736 raft_consensus.cc:2454] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec4126c1abf498e943da54ea79acc13 in term 1.
I20260809 13:12:38.677510 2153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" is_pre_election: true
I20260809 13:12:38.677821 2153 raft_consensus.cc:2379] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3efcbbad864446bc90aadaf6ed2a087c in current term 1: Already voted for candidate 0ec4126c1abf498e943da54ea79acc13 in this term.
I20260809 13:12:38.677856 1955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5733cd5adb1d44228b27c397d386b7bb" candidate_uuid: "3efcbbad864446bc90aadaf6ed2a087c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec4126c1abf498e943da54ea79acc13" is_pre_election: true
I20260809 13:12:38.678849 1636 leader_election.cc:302] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3efcbbad864446bc90aadaf6ed2a087c; no voters: 0ec4126c1abf498e943da54ea79acc13, 9a0ec46d2ced41a68fbae8fe0db0ea86
I20260809 13:12:38.679224 2253 raft_consensus.cc:2726] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 13:12:38.703987 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.699114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.10.181.130:43231)
0809 13:12:38.699295 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.699301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.699400 (+ 99us) server_negotiation.cc:407] Connection header received
0809 13:12:38.699579 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.699583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.699659 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.699772 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:12:38.700072 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.700775 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.702169 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.702573 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.702664 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.703067 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:12:38.703517 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:12:38.703621 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.703661 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.703722 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:12:38.711251 2153 raft_consensus.cc:1261] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Refusing update from remote peer 0ec4126c1abf498e943da54ea79acc13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:38.711299 1736 raft_consensus.cc:1261] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Refusing update from remote peer 0ec4126c1abf498e943da54ea79acc13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:12:38.712069 2256 consensus_queue.cc:1035] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:12:38.712333 2265 consensus_queue.cc:1035] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3efcbbad864446bc90aadaf6ed2a087c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:12:38.717370 2224 tablet.cc:2369] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:12:38.724807 2272 mvcc.cc:205] Tried to move back new op lower bound from 7316607626072850432 to 7316607625919840256. Current Snapshot: MvccSnapshot[applied={T|T < 7316607626072850432}]
I20260809 13:12:38.737054 1521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:38.737257 1521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:12:38.740556 1521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Sending DeleteTablet for 3 replicas of tablet 5733cd5adb1d44228b27c397d386b7bb
I20260809 13:12:38.741690 1922 tablet_service.cc:1514] Processing DeleteTablet for tablet 5733cd5adb1d44228b27c397d386b7bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:38 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:45900
I20260809 13:12:38.741705 1705 tablet_service.cc:1514] Processing DeleteTablet for tablet 5733cd5adb1d44228b27c397d386b7bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:38 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:52428
I20260809 13:12:38.742106 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 1577
I20260809 13:12:38.742712 2127 tablet_service.cc:1514] Processing DeleteTablet for tablet 5733cd5adb1d44228b27c397d386b7bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:12:38 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:49242
I20260809 13:12:38.743918 2282 tablet_replica.cc:330] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: stopping tablet replica
I20260809 13:12:38.744308 2282 raft_consensus.cc:2229] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:38.744760 2282 raft_consensus.cc:2258] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:38.745813 2283 tablet_replica.cc:330] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: stopping tablet replica
I20260809 13:12:38.746209 2283 raft_consensus.cc:2229] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:12:38.748338 2282 ts_tablet_manager.cc:1905] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:38.749004 2281 tablet_replica.cc:330] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: stopping tablet replica
I20260809 13:12:38.749555 2283 raft_consensus.cc:2258] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:38.750046 2281 raft_consensus.cc:2229] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:12:38.750763 2281 raft_consensus.cc:2258] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:12:38.752969 2281 ts_tablet_manager.cc:1905] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:38.753013 2283 ts_tablet_manager.cc:1905] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:38.760317 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.743002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.10.181.129:42067)
0809 13:12:38.743161 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.743168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.743213 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:12:38.743385 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.743390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.743465 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.743566 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:38.745354 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.746536 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.748526 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.749033 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.749156 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.752656 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:38.752670 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:38.752684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:38.752737 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:38.756300 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.757181 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.757191 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.757195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.757350 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.757831 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.757836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.757840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.758267 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:38.758572 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.759712 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.759918 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:12:38.760486 2281 ts_tablet_manager.cc:1918] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:38.760675 2281 log.cc:1196] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/5733cd5adb1d44228b27c397d386b7bb
I20260809 13:12:38.761221 1800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:38.761350 2281 ts_tablet_manager.cc:1939] T 5733cd5adb1d44228b27c397d386b7bb P 0ec4126c1abf498e943da54ea79acc13: Deleting consensus metadata
I20260809 13:12:38.762276 2283 ts_tablet_manager.cc:1918] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:38.762426 2283 log.cc:1196] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/5733cd5adb1d44228b27c397d386b7bb
I20260809 13:12:38.762967 2283 ts_tablet_manager.cc:1939] T 5733cd5adb1d44228b27c397d386b7bb P 9a0ec46d2ced41a68fbae8fe0db0ea86: Deleting consensus metadata
I20260809 13:12:38.763275 1499 catalog_manager.cc:4841] TS 0ec4126c1abf498e943da54ea79acc13 (127.10.181.130:43231): tablet 5733cd5adb1d44228b27c397d386b7bb (table test-table [id=47e46c9735bd42989a43833bcb49d919]) successfully deleted
I20260809 13:12:38.764657 1499 catalog_manager.cc:4841] TS 9a0ec46d2ced41a68fbae8fe0db0ea86 (127.10.181.131:41315): tablet 5733cd5adb1d44228b27c397d386b7bb (table test-table [id=47e46c9735bd42989a43833bcb49d919]) successfully deleted
I20260809 13:12:38.883354 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 1577
W20260809 13:12:38.908790 1501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:42067 (error 108)
I20260809 13:12:38.909758 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 1815
I20260809 13:12:38.911015 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.910420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52450)
0809 13:12:38.910707 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.910829 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260809 13:12:38.911281 1501 catalog_manager.cc:4584] TS 3efcbbad864446bc90aadaf6ed2a087c (127.10.181.129:42067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5733cd5adb1d44228b27c397d386b7bb: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
I20260809 13:12:38.925376 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.910535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.10.181.130:43231)
0809 13:12:38.910768 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:38.910774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:38.910890 (+ 116us) server_negotiation.cc:407] Connection header received
0809 13:12:38.911095 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:38.911101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:38.911191 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:38.911296 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:38.912696 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.913949 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.915242 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:38.915947 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:38.916083 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:38.918713 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:38.918731 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:38.918750 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:38.918802 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:38.922631 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.923292 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.923301 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.923305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.923465 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:38.923797 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:38.923802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:38.923806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:38.924311 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:38.924650 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:38.924801 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 13:12:38.925038 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 13:12:38.926080 2020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:38.964998 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.964425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52458)
0809 13:12:38.964680 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.964806 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 13:12:38.966058 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:38.965665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52468)
0809 13:12:38.965823 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:38.965923 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 13:12:39.026480 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.025893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52482)
0809 13:12:39.026141 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:39.026264 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 13:12:39.027544 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.027156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52484)
0809 13:12:39.027318 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:39.027405 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:12:39.040040 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 1815
I20260809 13:12:39.066795 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 2029
I20260809 13:12:39.082589 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.067533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.10.181.131:41315)
0809 13:12:39.067723 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:39.067728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:39.067748 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:12:39.067884 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:39.067888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:39.067968 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:39.068071 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:39.069326 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:39.070031 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:39.071210 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:39.071602 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:39.071692 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:39.077325 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:39.077343 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:39.077373 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:39.077412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:39.080148 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:39.080789 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:39.080796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:39.080799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:39.080914 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:39.081180 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:39.081183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:39.081185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:39.081610 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:39.081832 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:39.082178 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 13:12:39.082344 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:12:39.083400 2218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:39.107678 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.107178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52486)
0809 13:12:39.107392 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:39.107496 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:12:39.108714 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.108271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52502)
0809 13:12:39.108481 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:39.108570 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 13:12:39.174223 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 2029
I20260809 13:12:39.204217 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 1466
I20260809 13:12:39.205382 1569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:39.317382 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:39.316720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42067 (local address 127.0.0.1:52512)
0809 13:12:39.316920 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:39.317040 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 13:12:39.317543 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 1466
I20260809 13:12:39.346525 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--webserver_interface=127.10.181.190
--webserver_port=42369
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:39.675885 2293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:39.676168 2293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:39.676297 2293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:39.683328 2293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:39.683423 2293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:39.683495 2293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:39.683570 2293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:39.683629 2293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:39.692868 2293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:32925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=42369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:39.694068 2293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:39.695264 2293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:39.706297 2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:39.706256 2302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:39.706743 2306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:39.707118 2293 server_base.cc:1034] running on GCE node
I20260809 13:12:39.707847 2293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:39.709182 2293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:39.710389 2293 hybrid_clock.cc:648] HybridClock initialized: now 1786281159710328 us; error 50 us; skew 500 ppm
I20260809 13:12:39.710724 2293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:39 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281159, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:39.713652 2293 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:39.715029 2293 webserver.cc:460] Webserver started at http://127.10.181.190:42369/ using document root <none> and password file <none>
I20260809 13:12:39.715669 2293 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:39.715787 2293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:39.720973 2293 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:12:39.723879 2315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:39.725145 2293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:12:39.725306 2293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "fb9f9e2bb4ee411fabea61654aaac7be"
format_stamp: "Formatted at 2026-08-09 13:12:33 on dist-test-slave-42z9"
server_key: "71d0ef68e78315ebdb95b69ccb30aad7"
server_key_iv: "346f2d756e766384e9e558b85e3effd1"
server_key_version: "encryptionkey@0"
I20260809 13:12:39.726277 2293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:39.744132 2293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:39.747824 2293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:39.748075 2293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:39.765513 2293 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:32925
I20260809 13:12:39.765537 2393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:32925 every 8 connection(s)
I20260809 13:12:39.767020 2293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:39.770937 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 2293
I20260809 13:12:39.771937 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:42067
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=42613
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:39.775055 2396 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:12:39.779239 2396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap starting.
I20260809 13:12:39.800087 2396 log.cc:824] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:39.816010 2396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:12:39.816516 2396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be: Bootstrap complete.
I20260809 13:12:39.825292 2396 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:39.826180 2396 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Initialized, Role: FOLLOWER
I20260809 13:12:39.826814 2396 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:39.827026 2396 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:39.827135 2396 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:39.827296 2396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 2 FOLLOWER]: Advancing to term 3
I20260809 13:12:39.831137 2396 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:39.831594 2396 leader_election.cc:302] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb9f9e2bb4ee411fabea61654aaac7be; no voters:
I20260809 13:12:39.832203 2396 leader_election.cc:288] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 13:12:39.832342 2403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 3 FOLLOWER]: Leader election won for term 3
I20260809 13:12:39.833433 2403 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [term 3 LEADER]: Becoming Leader. State: Replica: fb9f9e2bb4ee411fabea61654aaac7be, State: Running, Role: LEADER
I20260809 13:12:39.834061 2403 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } }
I20260809 13:12:39.834306 2396 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:39.837924 2407 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb9f9e2bb4ee411fabea61654aaac7be. Latest consensus state: current_term: 3 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:39.838979 2407 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:39.838394 2404 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9f9e2bb4ee411fabea61654aaac7be" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 32925 } } }
I20260809 13:12:39.839418 2404 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9f9e2bb4ee411fabea61654aaac7be [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:39.850922 2416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:39.859243 2416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:39.860810 2416 catalog_manager.cc:1259] Loaded cluster ID: 52f1dffeabf14fbc8cb2c2cf0612382b
I20260809 13:12:39.860956 2416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 13:12:39.863166 2416 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe7afbd79ad google::LogMessage::Fail() at ??:0
@ 0x7fe7afbda88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe7afbd74de google::LogMessage::Flush() at ??:0
@ 0x7fe7afbd8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe7c560a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe7c5607976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe7c568f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe7c568f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe7b2823345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe7b282691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe7b2826799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe7b2801eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe7b5adb6db start_thread at ??:0
@ 0x7fe7aea7161f clone at ??:0
W20260809 13:12:40.165717 2400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:40.165984 2400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:40.166121 2400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:40.173241 2400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:40.173357 2400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:40.173537 2400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:40.183596 2400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:42067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=42613
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:40.184844 2400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:40.186221 2400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:40.199010 2453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:40.198923 2456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:40.198978 2454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:40.200142 2400 server_base.cc:1034] running on GCE node
I20260809 13:12:40.200702 2400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:40.201570 2400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:40.202764 2400 hybrid_clock.cc:648] HybridClock initialized: now 1786281160202716 us; error 42 us; skew 500 ppm
I20260809 13:12:40.203032 2400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:40 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281160, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:40.206092 2400 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:40.207463 2400 webserver.cc:460] Webserver started at http://127.10.181.129:42613/ using document root <none> and password file <none>
I20260809 13:12:40.208086 2400 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:40.208187 2400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:40.213260 2400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:40.218204 2468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:40.219475 2400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:40.219619 2400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "3efcbbad864446bc90aadaf6ed2a087c"
format_stamp: "Formatted at 2026-08-09 13:12:34 on dist-test-slave-42z9"
server_key: "c2b59ea444c326fa0211e6e289cdbf30"
server_key_iv: "f94b6f7606c19daf5c4d6e55e08793ec"
server_key_version: "encryptionkey@0"
I20260809 13:12:40.220511 2400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:40.239367 2400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:40.243417 2400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:40.243644 2400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:40.244565 2400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:40.246342 2478 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 13:12:40.251751 2478 ts_tablet_manager.cc:1852] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 13:12:40.252769 2478 ts_tablet_manager.cc:1905] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:12:40.259996 2478 ts_tablet_manager.cc:1918] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:12:40.260205 2478 log.cc:1196] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/5733cd5adb1d44228b27c397d386b7bb
I20260809 13:12:40.260653 2478 ts_tablet_manager.cc:1939] T 5733cd5adb1d44228b27c397d386b7bb P 3efcbbad864446bc90aadaf6ed2a087c: Deleting consensus metadata
I20260809 13:12:40.261020 2400 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 13:12:40.261125 2400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260809 13:12:40.261245 2400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:40.261330 2400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:40.292603 2400 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:42067
I20260809 13:12:40.292652 2635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:42067 every 8 connection(s)
I20260809 13:12:40.293936 2400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:40.298614 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.296766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:38009)
0809 13:12:40.297538 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.297784 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":261,"threads_started":1}
W20260809 13:12:40.299193 2636 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:32925 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:32925: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
I20260809 13:12:40.299909 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.299628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:37683)
0809 13:12:40.299747 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.299811 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 13:12:40.300642 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.300361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:50303)
0809 13:12:40.300464 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.300532 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 13:12:40.300830 2636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:12:40.303323 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 2400
I20260809 13:12:40.303953 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:43231
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=42841
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:40.636986 2643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:40.637239 2643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:40.637336 2643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:40.644297 2643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:40.644399 2643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:40.644513 2643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:40.653805 2643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:43231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=42841
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:40.655241 2643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:40.656724 2643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:40.669139 2649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:40.669096 2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:40.669179 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:40.669937 2643 server_base.cc:1034] running on GCE node
I20260809 13:12:40.670506 2643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:40.671365 2643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:40.672559 2643 hybrid_clock.cc:648] HybridClock initialized: now 1786281160672510 us; error 32 us; skew 500 ppm
I20260809 13:12:40.672845 2643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:40 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281160, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:40.675791 2643 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:40.677165 2643 webserver.cc:460] Webserver started at http://127.10.181.130:42841/ using document root <none> and password file <none>
I20260809 13:12:40.677843 2643 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:40.677958 2643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:40.683058 2643 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:12:40.685602 2660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:40.686825 2643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:12:40.686975 2643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "0ec4126c1abf498e943da54ea79acc13"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "2f4444c5cc4718d552a2ba6a5cbf63d7"
server_key_iv: "674c7c602c4490c252936dc42a646fee"
server_key_version: "encryptionkey@0"
I20260809 13:12:40.687896 2643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:40.698310 2643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:40.701807 2643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:40.702035 2643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:40.702944 2643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:40.704161 2643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:40.704258 2643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:40.704360 2643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:40.704428 2643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:40.735249 2643 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:43231
I20260809 13:12:40.735318 2837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:43231 every 8 connection(s)
I20260809 13:12:40.736658 2643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:40.740585 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 2643
I20260809 13:12:40.741377 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:41315
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:32925
--webserver_port=35037
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:12:40.741652 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.739443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:54733)
0809 13:12:40.740170 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.740448 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":263,"threads_started":1}
W20260809 13:12:40.742336 2838 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:32925 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:32925: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
I20260809 13:12:40.743160 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.742799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:36359)
0809 13:12:40.742949 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.743032 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:12:40.744001 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:40.743672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:52639)
0809 13:12:40.743799 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:40.743883 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 13:12:40.744230 2838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 13:12:41.070596 2844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:41.070843 2844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:41.070973 2844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:41.078037 2844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:41.078142 2844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:41.078290 2844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:12:41.087834 2844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:41315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=35037
--tserver_master_addrs=127.10.181.190:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:41.088994 2844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:41.090261 2844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:41.102813 2850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:41.102775 2851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:41.102735 2854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:41.103827 2844 server_base.cc:1034] running on GCE node
I20260809 13:12:41.104404 2844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:41.105259 2844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:41.106460 2844 hybrid_clock.cc:648] HybridClock initialized: now 1786281161106408 us; error 43 us; skew 500 ppm
I20260809 13:12:41.106778 2844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:41 dist-test-slave-42z9 krb5kdc[596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281161, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:41.109802 2844 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:12:41.111084 2844 webserver.cc:460] Webserver started at http://127.10.181.131:35037/ using document root <none> and password file <none>
I20260809 13:12:41.111718 2844 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:41.111817 2844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:41.116827 2844 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:12:41.119419 2866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:41.120694 2844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:41.120850 2844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "9a0ec46d2ced41a68fbae8fe0db0ea86"
format_stamp: "Formatted at 2026-08-09 13:12:35 on dist-test-slave-42z9"
server_key: "c5ec222c260ba7a1a96a7abb31d63b87"
server_key_iv: "34045551ecdbab37a30593c86a8b120e"
server_key_version: "encryptionkey@0"
I20260809 13:12:41.121819 2844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:41.134645 2844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:41.138293 2844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:41.138526 2844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:41.139477 2844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:41.140702 2844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:41.140791 2844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:41.140877 2844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:41.140933 2844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:41.175531 2844 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41315
I20260809 13:12:41.175601 3032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41315 every 8 connection(s)
I20260809 13:12:41.177302 2844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:12:41.178763 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 2844
I20260809 13:12:41.183578 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:41.181286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:58119)
0809 13:12:41.182331 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:41.182593 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":735,"threads_started":1}
W20260809 13:12:41.184224 3033 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:32925 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:32925: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
I20260809 13:12:41.186887 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:41.184779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:56405)
0809 13:12:41.186700 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:41.186775 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1821}
I20260809 13:12:41.188536 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:41.188247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:41885)
0809 13:12:41.188364 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:41.188434 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 13:12:41.188766 3033 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:12:41.302184 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:41.301358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:55357)
0809 13:12:41.301861 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:41.301959 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":261,"threads_started":1}
I20260809 13:12:41.745513 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:41.744735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:58367)
0809 13:12:41.745164 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:41.745269 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":201,"threads_started":1}
I20260809 13:12:42.190076 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:42.189253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:40059)
0809 13:12:42.189732 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:42.189853 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260809 13:12:42.303614 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:42.302971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:54101)
0809 13:12:42.303389 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:42.303468 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":223,"threads_started":1}
I20260809 13:12:42.747014 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:42.746304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:35431)
0809 13:12:42.746764 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:42.746848 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":217,"threads_started":1}
I20260809 13:12:43.191679 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:43.190899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:52703)
0809 13:12:43.191414 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:43.191497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":289,"threads_started":1}
I20260809 13:12:43.305148 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:43.304458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:44963)
0809 13:12:43.304899 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:43.304982 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":220,"threads_started":1}
I20260809 13:12:43.748521 3053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:43.747825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:45029)
0809 13:12:43.748272 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:43.748354 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":233,"threads_started":1}
I20260809 13:12:44.193188 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:44.192469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:38933)
0809 13:12:44.192934 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:44.193018 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":246,"threads_started":1}
I20260809 13:12:44.306558 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:44.305903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:49067)
0809 13:12:44.306331 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:44.306410 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260809 13:12:44.750010 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:44.749273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:59509)
0809 13:12:44.749743 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:44.749823 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260809 13:12:45.194763 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:45.194027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:57387)
0809 13:12:45.194479 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:45.194563 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":216,"threads_started":1}
I20260809 13:12:45.308152 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:45.307436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:55139)
0809 13:12:45.307878 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:45.307960 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260809 13:12:45.751518 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:45.750816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:40955)
0809 13:12:45.751267 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:45.751350 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":220,"threads_started":1}
I20260809 13:12:46.196378 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:46.195626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:50755)
0809 13:12:46.196127 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:46.196211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260809 13:12:46.309691 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:46.308979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:53187)
0809 13:12:46.309402 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:46.309518 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":213,"threads_started":1}
I20260809 13:12:46.753036 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:46.752332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:59275)
0809 13:12:46.752776 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:46.752866 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260809 13:12:47.197988 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:47.197223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:34489)
0809 13:12:47.197729 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:47.197820 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
I20260809 13:12:47.311370 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:47.310567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:57561)
0809 13:12:47.311070 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:47.311169 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":212,"threads_started":1}
I20260809 13:12:47.754768 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:47.753974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:56489)
0809 13:12:47.754459 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:47.754545 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":211,"threads_started":1}
I20260809 13:12:48.199653 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:48.198821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:37537)
0809 13:12:48.199337 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:48.199429 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":205,"threads_started":1}
I20260809 13:12:48.312803 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:48.312149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:49791)
0809 13:12:48.312549 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:48.312641 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":202,"threads_started":1}
I20260809 13:12:48.756247 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:48.755563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:54191)
0809 13:12:48.755992 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:48.756074 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":219,"threads_started":1}
I20260809 13:12:49.201179 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:49.200497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:55143)
0809 13:12:49.200933 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:49.201013 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":226,"threads_started":1}
I20260809 13:12:49.314345 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:49.313610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:59681)
0809 13:12:49.314055 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:49.314164 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":208,"threads_started":1}
I20260809 13:12:49.757797 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:49.757018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:39419)
0809 13:12:49.757527 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:49.757612 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":224,"threads_started":1}
I20260809 13:12:50.202629 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:50.201926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:45083)
0809 13:12:50.202378 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:50.202461 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260809 13:12:50.315971 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:50.315307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:33627)
0809 13:12:50.315727 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:50.315809 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":209,"threads_started":1}
I20260809 13:12:50.759251 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:50.758598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:39099)
0809 13:12:50.759020 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:50.759101 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":222,"threads_started":1}
I20260809 13:12:51.204187 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:51.203459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:53695)
0809 13:12:51.203941 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:51.204024 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":261,"threads_started":1}
I20260809 13:12:51.317510 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:51.316763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:47661)
0809 13:12:51.317203 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:51.317284 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":229,"threads_started":1}
I20260809 13:12:51.760727 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:51.760040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:59877)
0809 13:12:51.760472 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:51.760553 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":214,"threads_started":1}
I20260809 13:12:52.205785 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:52.205016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:44237)
0809 13:12:52.205481 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:52.205588 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":221,"threads_started":1}
I20260809 13:12:52.319051 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:52.318306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:45295)
0809 13:12:52.318812 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:52.318896 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":276,"threads_started":1}
I20260809 13:12:52.762215 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:52.761564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:50361)
0809 13:12:52.761974 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:52.762053 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":196,"threads_started":1}
I20260809 13:12:53.207331 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:53.206639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:55479)
0809 13:12:53.207066 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:53.207145 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":220,"threads_started":1}
I20260809 13:12:53.320721 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:53.319886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:53087)
0809 13:12:53.320327 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:53.320456 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":219,"threads_started":1}
I20260809 13:12:53.763784 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:53.763016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:56067)
0809 13:12:53.763495 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:53.763607 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":237,"threads_started":1}
I20260809 13:12:54.208900 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:54.208178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:42569)
0809 13:12:54.208620 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:54.208712 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":212,"threads_started":1}
I20260809 13:12:54.322278 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:54.321581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:56569)
0809 13:12:54.322004 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:54.322089 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":206,"threads_started":1}
I20260809 13:12:54.765285 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:54.764565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:46423)
0809 13:12:54.765018 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:54.765103 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":214,"threads_started":1}
I20260809 13:12:55.210419 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:55.209737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:57441)
0809 13:12:55.210152 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:55.210233 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":201,"threads_started":1}
I20260809 13:12:55.324187 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:55.323372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.129:41495)
0809 13:12:55.323897 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:55.324002 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":255,"threads_started":1}
I20260809 13:12:55.767091 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:55.766333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.130:53825)
0809 13:12:55.766798 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:55.766907 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":228,"threads_started":1}
I20260809 13:12:56.185631 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 2400
I20260809 13:12:56.203763 3108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:56.186821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.10.181.129:42067)
0809 13:12:56.187494 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:56.187509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:56.187554 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:12:56.189592 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:56.189605 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:56.190052 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:56.190378 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:56.191786 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.193200 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.194478 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.195147 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.195281 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:56.197975 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:56.197983 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:56.197997 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:56.198041 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:56.200677 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.201367 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.201375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.201381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.201615 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.201949 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.201954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.201956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.202523 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:56.202767 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:56.203125 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 13:12:56.203471 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":369,"threads_started":1}
I20260809 13:12:56.205205 2627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:56.212255 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:56.211498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:32925 (local address 127.10.181.131:45631)
0809 13:12:56.211971 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:56.212077 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:32925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":239,"threads_started":1}
I20260809 13:12:56.292274 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 2400
I20260809 13:12:56.322372 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 2643
I20260809 13:12:56.345711 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:56.323353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.10.181.130:43231)
0809 13:12:56.323969 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:56.323989 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:56.324035 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:12:56.325879 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:56.325892 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:56.326383 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:56.326800 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:56.328314 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.329665 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.330888 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.331537 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.331672 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:56.339791 (+ 8119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:56.339799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:56.339815 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:56.339862 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:56.342873 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.343484 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.343491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.343497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.343670 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.343942 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.343945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.343947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.344464 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:56.344725 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:56.345020 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 13:12:56.345370 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":273,"threads_started":1}
I20260809 13:12:56.347168 2827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:56.431825 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 2643
I20260809 13:12:56.457047 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 2844
I20260809 13:12:56.474078 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:56.457988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.10.181.131:41315)
0809 13:12:56.458609 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:56.458630 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:56.458678 (+ 48us) server_negotiation.cc:407] Connection header received
0809 13:12:56.460451 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:56.460463 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:56.460913 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:56.461199 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:56.462588 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.463815 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.464993 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:56.465706 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:56.465844 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:56.468301 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:56.468309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:56.468324 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:56.468363 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:56.471184 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.471780 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.471787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.471793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.471961 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:56.472252 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:56.472255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:56.472257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:56.472786 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:56.473033 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:56.473333 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 13:12:56.473777 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":282,"threads_started":1}
I20260809 13:12:56.475421 3025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:12:56.559263 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 2844
2026-08-09T13:12:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24498 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:12:56 dist-test-slave-42z9 krb5kdc[3122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:12:56 dist-test-slave-42z9 krb5kdc[3122](info): set up 2 sockets
Aug 09 13:12:56 dist-test-slave-42z9 krb5kdc[3122](info): commencing operation
krb5kdc: starting...
W20260809 13:12:57.796862 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:12:57 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:12:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:12:58.068655 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:58.413412 3139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:58.413710 3139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:58.413807 3139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:58.420900 3139 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:12:58.421005 3139 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:12:58.421051 3139 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:12:58.421101 3139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:12:58.421154 3139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:58.421221 3139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:58.421265 3139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:12:58.421319 3139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:12:58.431210 3139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:58.432505 3139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:58.433859 3139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:58.445267 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:58.445327 3153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:58.445546 3150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:58.446099 3139 server_base.cc:1034] running on GCE node
I20260809 13:12:58.446815 3139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:58.448136 3139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:58.449373 3139 hybrid_clock.cc:648] HybridClock initialized: now 1786281178449312 us; error 26 us; skew 500 ppm
I20260809 13:12:58.449731 3139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:58 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281178, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:58.452692 3139 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:12:58.454131 3139 webserver.cc:460] Webserver started at http://127.10.181.190:38427/ using document root <none> and password file <none>
I20260809 13:12:58.454803 3139 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:58.454905 3139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:58.455211 3139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:58.458185 3139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "739eb820ecf741eab6261d0f16815c58"
format_stamp: "Formatted at 2026-08-09 13:12:58 on dist-test-slave-42z9"
server_key: "669b3bb157e30ed433076903b2290b30"
server_key_iv: "ea18b7c3b467071bb63d0869fffe1584"
server_key_version: "encryptionkey@0"
I20260809 13:12:58.458938 3139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "739eb820ecf741eab6261d0f16815c58"
format_stamp: "Formatted at 2026-08-09 13:12:58 on dist-test-slave-42z9"
server_key: "669b3bb157e30ed433076903b2290b30"
server_key_iv: "ea18b7c3b467071bb63d0869fffe1584"
server_key_version: "encryptionkey@0"
I20260809 13:12:58.464015 3139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 13:12:58.467407 3163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:58.468696 3139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:58.468859 3139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "739eb820ecf741eab6261d0f16815c58"
format_stamp: "Formatted at 2026-08-09 13:12:58 on dist-test-slave-42z9"
server_key: "669b3bb157e30ed433076903b2290b30"
server_key_iv: "ea18b7c3b467071bb63d0869fffe1584"
server_key_version: "encryptionkey@0"
I20260809 13:12:58.469060 3139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:58.481410 3139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:58.485095 3139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:58.485335 3139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:58.502738 3139 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46013
I20260809 13:12:58.502759 3240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46013 every 8 connection(s)
I20260809 13:12:58.504063 3139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:12:58.505060 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 3139
I20260809 13:12:58.505293 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:12:58.506011 10966 external_mini_cluster.cc:1428] Setting key 4cb1119b7dc924fe192d43299803211a
I20260809 13:12:58.508327 3241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:12:58 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:58.521502 3241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap starting.
I20260809 13:12:58.525578 3241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Neither blocks nor log segments found. Creating new log.
I20260809 13:12:58.526854 3241 log.cc:824] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Log is configured to *not* fsync() on all Append() calls
I20260809 13:12:58.530417 3241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: No bootstrap required, opened a new log
I20260809 13:12:58.530556 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:58.508042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.10.181.190:46013)
0809 13:12:58.508874 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:58.508889 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:58.508940 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:12:58.511187 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:58.511215 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:58.511709 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:58.512126 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:58.513472 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:58.515509 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:58.516940 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:58.517986 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:58.518191 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:58.522408 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:58.522419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:58.522437 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:58.522488 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:58.526287 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:58.526952 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:58.526961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:58.526968 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:58.527193 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:58.527597 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:58.527602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:58.527606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:58.528144 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:58.528425 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:58.528840 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 13:12:58.529265 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":351,"threads_started":1}
I20260809 13:12:58.538298 3241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:12:58.538637 3241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:12:58.538712 3241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Initialized, Role: FOLLOWER
I20260809 13:12:58.539206 3241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:12:58.539438 3241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:12:58.539546 3241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:12:58.539685 3241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:12:58.542052 3241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:12:58.542503 3241 leader_election.cc:302] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 739eb820ecf741eab6261d0f16815c58; no voters:
I20260809 13:12:58.543041 3241 leader_election.cc:288] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:12:58.543157 3246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:12:58.543833 3246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 LEADER]: Becoming Leader. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Running, Role: LEADER
I20260809 13:12:58.544386 3246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:12:58.544992 3241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:12:58.547274 3247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:12:58.548239 3247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:58.547755 3248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 739eb820ecf741eab6261d0f16815c58. Latest consensus state: current_term: 1 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:12:58.548719 3248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:12:58.550307 3254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:12:58.555558 3254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:12:58.564175 3254 catalog_manager.cc:1347] Generated new cluster ID: 46d4f6ceb891452298639642c4373e21
I20260809 13:12:58.564281 3254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:12:58.592082 3254 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:12:58.593914 3254 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:12:58.605269 3254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Generated new TSK 0
I20260809 13:12:58.606287 3254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:12:58.689337 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:59.035894 3276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:59.036185 3276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:59.036324 3276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:59.043812 3276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:59.043936 3276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:59.044085 3276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:12:59.054175 3276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:59.055524 3276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:59.057013 3276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:59.069984 3282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:59.070010 3283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:59.070060 3287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:59.070971 3276 server_base.cc:1034] running on GCE node
I20260809 13:12:59.071645 3276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:59.072618 3276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:59.073828 3276 hybrid_clock.cc:648] HybridClock initialized: now 1786281179073768 us; error 54 us; skew 500 ppm
I20260809 13:12:59.074146 3276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:59 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:59.077198 3276 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:12:59.078648 3276 webserver.cc:460] Webserver started at http://127.10.181.129:33139/ using document root <none> and password file <none>
I20260809 13:12:59.079366 3276 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:59.079485 3276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:59.079742 3276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:59.082679 3276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "1340000bd0c0123d250df58848144eb2"
server_key_iv: "7760292ad755e60663d558721ae87e1c"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.083421 3276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "1340000bd0c0123d250df58848144eb2"
server_key_iv: "7760292ad755e60663d558721ae87e1c"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.088507 3276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 13:12:59.091950 3298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.093256 3276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:12:59.093415 3276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "1340000bd0c0123d250df58848144eb2"
server_key_iv: "7760292ad755e60663d558721ae87e1c"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.093657 3276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:59.130251 3276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:59.134302 3276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:59.134585 3276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:59.135608 3276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:59.136951 3276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:59.137040 3276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.137161 3276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:59.137215 3276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.179180 3276 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:40953
I20260809 13:12:59.179241 3462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:40953 every 8 connection(s)
I20260809 13:12:59.180488 3276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:12:59.180821 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 3276
I20260809 13:12:59.181015 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:12:59.181921 10966 external_mini_cluster.cc:1428] Setting key 396a2a21faea38170f27dfa2623e6498
Aug 09 13:12:59 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:59.199838 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:59.183117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54085 (local address 127.10.181.190:46013)
0809 13:12:59.183601 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:59.183607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:59.184636 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 13:12:59.186485 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:59.186490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:59.186572 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:59.186669 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:59.188169 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.188908 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.190466 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.190872 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.190985 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:59.194291 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:59.194297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:59.194300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:59.194347 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:59.196801 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:59.197538 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:59.197544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:59.197548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:59.197674 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:59.198104 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:59.198109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:59.198113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:59.198443 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:59.198581 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:59.199252 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 13:12:59.199479 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260809 13:12:59.200536 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:59.183725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:54085)
0809 13:12:59.184476 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:59.184522 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:59.186229 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:59.186899 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:59.186907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:59.187425 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:59.188002 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:59.188020 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.189063 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.189066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:59.190325 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:59.190339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.191097 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.191105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:59.191337 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:59.191950 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:59.191973 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:59.194083 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:59.196919 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:59.196928 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:59.196944 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:59.197399 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:59.197792 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:59.197795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:59.197798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:59.197978 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:59.198566 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:59.198572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:59.198855 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 13:12:59.199093 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 13:12:59.199482 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":306,"threads_started":1}
I20260809 13:12:59.202091 3463 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:12:59.202414 3463 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:59.203294 3463 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:12:59.205590 3188 ts_manager.cc:194] Registered new tserver with Master: 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953)
I20260809 13:12:59.207063 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:54085
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:12:59.248814 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:12:59.591863 3470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:12:59.592130 3470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:12:59.592229 3470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:12:59.599421 3470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:12:59.599529 3470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:12:59.599655 3470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:12:59.609134 3470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:12:59.610374 3470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:12:59.611716 3470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:12:59.624379 3482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:59.624498 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:12:59.624527 3478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:12:59.625566 3470 server_base.cc:1034] running on GCE node
I20260809 13:12:59.626200 3470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:12:59.627096 3470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:12:59.628310 3470 hybrid_clock.cc:648] HybridClock initialized: now 1786281179628249 us; error 48 us; skew 500 ppm
I20260809 13:12:59.628609 3470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:12:59 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:12:59.631634 3470 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:12:59.632978 3470 webserver.cc:460] Webserver started at http://127.10.181.130:42111/ using document root <none> and password file <none>
I20260809 13:12:59.633702 3470 fs_manager.cc:359] Metadata directory not provided
I20260809 13:12:59.633826 3470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:12:59.634130 3470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:12:59.637017 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "d137f235131042d1ae86e584badcf616"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "3ff2892140d26ba74b833e887746ed35"
server_key_iv: "d52481948d8c379c5e376beafa774cea"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.637804 3470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "d137f235131042d1ae86e584badcf616"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "3ff2892140d26ba74b833e887746ed35"
server_key_iv: "d52481948d8c379c5e376beafa774cea"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.643523 3470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:12:59.647238 3489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.648550 3470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:12:59.648705 3470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "d137f235131042d1ae86e584badcf616"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "3ff2892140d26ba74b833e887746ed35"
server_key_iv: "d52481948d8c379c5e376beafa774cea"
server_key_version: "encryptionkey@0"
I20260809 13:12:59.648891 3470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:12:59.685774 3470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:12:59.689868 3470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:12:59.690119 3470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:12:59.691075 3470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:12:59.692484 3470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:12:59.692570 3470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.692677 3470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:12:59.692728 3470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:12:59.725538 3470 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33033
I20260809 13:12:59.725567 3656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33033 every 8 connection(s)
I20260809 13:12:59.726917 3470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:12:59.728538 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 3470
I20260809 13:12:59.728705 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:12:59.729146 10966 external_mini_cluster.cc:1428] Setting key 15d8a30b6af8418d61a914a25d6cc71f
Aug 09 13:12:59 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:12:59.748157 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:59.729546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:58373 (local address 127.10.181.190:46013)
0809 13:12:59.730072 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 13:12:59.730078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:12:59.730937 (+ 859us) server_negotiation.cc:407] Connection header received
0809 13:12:59.733370 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:12:59.733374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:12:59.733462 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:12:59.733559 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:12:59.735508 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.736238 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.738298 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.738726 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.738814 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:59.742671 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:12:59.742676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:12:59.742680 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:12:59.742711 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:12:59.745148 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:59.745908 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:59.745912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:59.745915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:59.746078 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:12:59.746507 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:12:59.746511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:12:59.746513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:12:59.746778 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:12:59.746908 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 13:12:59.747566 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 13:12:59.747792 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":282,"threads_started":1}
I20260809 13:12:59.749014 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:12:59.730034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:58373)
0809 13:12:59.730781 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 13:12:59.730827 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:12:59.733041 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:12:59.733834 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:12:59.733844 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:12:59.734585 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:12:59.735351 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:59.735382 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.736388 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.736393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:59.738172 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:12:59.738189 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:12:59.738944 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:12:59.738950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:12:59.739243 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:12:59.740012 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:12:59.740042 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:12:59.742446 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:12:59.745283 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:59.745292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:59.745308 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:59.745787 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:59.746185 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:12:59.746189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:12:59.746193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:12:59.746390 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:12:59.746900 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:12:59.746907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:12:59.747217 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 13:12:59.747445 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 13:12:59.747937 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":307,"threads_started":1}
I20260809 13:12:59.750465 3657 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:12:59.750777 3657 heartbeater.cc:464] Registering TS with master...
I20260809 13:12:59.751471 3657 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:12:59.752964 3188 ts_manager.cc:194] Registered new tserver with Master: d137f235131042d1ae86e584badcf616 (127.10.181.130:33033)
I20260809 13:12:59.753816 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:58373
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:12:59.792615 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:13:00.133512 3666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:00.133819 3666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:00.133951 3666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:00.141021 3666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:00.141140 3666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:00.141302 3666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:13:00.151022 3666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:00.152324 3666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:00.153723 3666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:00.166288 3675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:00.166311 3679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:00.166443 3676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:00.167686 3666 server_base.cc:1034] running on GCE node
I20260809 13:13:00.168259 3666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:00.169148 3666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:00.170351 3666 hybrid_clock.cc:648] HybridClock initialized: now 1786281180170300 us; error 36 us; skew 500 ppm
I20260809 13:13:00.170671 3666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:00 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:00.173790 3666 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:13:00.175189 3666 webserver.cc:460] Webserver started at http://127.10.181.131:44301/ using document root <none> and password file <none>
I20260809 13:13:00.175853 3666 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:00.175964 3666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:00.176270 3666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:13:00.179618 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
format_stamp: "Formatted at 2026-08-09 13:13:00 on dist-test-slave-42z9"
server_key: "409b0ae1297dabe0df2ceaded0653255"
server_key_iv: "bdf03c78645ff3092d0934cf65e3048f"
server_key_version: "encryptionkey@0"
I20260809 13:13:00.180605 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
format_stamp: "Formatted at 2026-08-09 13:13:00 on dist-test-slave-42z9"
server_key: "409b0ae1297dabe0df2ceaded0653255"
server_key_iv: "bdf03c78645ff3092d0934cf65e3048f"
server_key_version: "encryptionkey@0"
I20260809 13:13:00.187062 3666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 13:13:00.190529 3689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:00.191788 3666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:13:00.191921 3666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
format_stamp: "Formatted at 2026-08-09 13:13:00 on dist-test-slave-42z9"
server_key: "409b0ae1297dabe0df2ceaded0653255"
server_key_iv: "bdf03c78645ff3092d0934cf65e3048f"
server_key_version: "encryptionkey@0"
I20260809 13:13:00.192090 3666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:00.210419 3463 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:00.213018 3666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:00.217417 3666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:00.217742 3666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:00.218736 3666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:00.220173 3666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:00.220259 3666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:00.220378 3666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:00.220436 3666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:00.251710 3666 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:34933
I20260809 13:13:00.251765 3856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:34933 every 8 connection(s)
I20260809 13:13:00.253110 3666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:13:00.253624 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 3666
I20260809 13:13:00.253832 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:13:00.254448 10966 external_mini_cluster.cc:1428] Setting key 6ab120cb035781caf506c0f4fa4f187f
Aug 09 13:13:00 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:13:00.273625 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.255992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60131 (local address 127.10.181.190:46013)
0809 13:13:00.256442 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.256448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.257499 (+ 1051us) server_negotiation.cc:407] Connection header received
0809 13:13:00.259461 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.259466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.259560 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.259654 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:00.261134 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.261987 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.263516 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.263927 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.264036 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.267398 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:00.267404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:00.267407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:00.267441 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:00.270685 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.271394 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.271400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.271404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.271538 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.271952 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.271958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.271962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.272231 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:00.272425 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.273082 (+ 657us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.273249 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":202,"threads_started":1}
I20260809 13:13:00.274530 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.256590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:60131)
0809 13:13:00.257335 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.257381 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.259206 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.259888 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.259896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.260417 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:13:00.260966 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.260983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.262140 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.262144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.263378 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.263391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.264152 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.264160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.264377 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.264961 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:13:00.264984 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:13:00.267214 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:13:00.270830 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:00.270839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:00.270857 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:00.271283 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:00.271642 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:00.271647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:00.271651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:00.271846 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:00.272417 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:13:00.272423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:13:00.272765 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.272999 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.273398 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":313,"threads_started":1}
I20260809 13:13:00.275740 3858 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:13:00.276032 3858 heartbeater.cc:464] Registering TS with master...
I20260809 13:13:00.276755 3858 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:13:00.278155 3188 ts_manager.cc:194] Registered new tserver with Master: ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933)
I20260809 13:13:00.278941 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:60131
I20260809 13:13:00.282500 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:13:00.298888 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.286981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.10.181.190:46013)
0809 13:13:00.287139 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.287143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.287651 (+ 508us) server_negotiation.cc:407] Connection header received
0809 13:13:00.287800 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.287803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.287860 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.287937 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:00.289405 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.290158 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.291448 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.291811 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.291894 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.294118 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:00.294123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:00.294127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:00.294156 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:00.296499 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.297130 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.297134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.297137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.297254 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.297682 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.297687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.297691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.298016 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:00.298196 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.298438 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.298618 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:13:00.302582 3188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:13:00.309372 3188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:13:00.345043 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.321362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:34933 (local address 127.0.0.1:34726)
0809 13:13:00.334886 (+ 13524us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.334941 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.335261 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.336385 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.336393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.336427 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:00.336817 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.336830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.338835 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.338840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.341339 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.341356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.343157 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.343163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.344131 (+ 968us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.344195 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.344322 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.344520 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13225,"mutex_wait_us":371,"thread_start_us":13079,"threads_started":1}
I20260809 13:13:00.345687 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.321738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.10.181.131:34933)
0809 13:13:00.334047 (+ 12309us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.334058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.335081 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 13:13:00.335529 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.335538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.335818 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.335996 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:00.337043 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.338649 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.341542 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.342964 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.343149 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.343335 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.345084 (+ 1749us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.345309 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12062,"thread_start_us":12327,"threads_started":1}
I20260809 13:13:00.347532 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.321949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.10.181.130:33033)
0809 13:13:00.338849 (+ 16900us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.338858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.339812 (+ 954us) server_negotiation.cc:407] Connection header received
0809 13:13:00.339932 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.339941 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.340182 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.340376 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:00.341297 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.342841 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.344999 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.346476 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.346645 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.346811 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.347090 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.347250 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16737,"thread_start_us":16638,"threads_started":1}
I20260809 13:13:00.347558 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.321841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33033 (local address 127.0.0.1:33494)
0809 13:13:00.339326 (+ 17485us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.339354 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.339486 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.340611 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.340616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.340641 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:00.341084 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.341096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.342973 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.342977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.344839 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.344857 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.346701 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.346708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.346887 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.346921 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.347010 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.347164 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17346,"mutex_wait_us":4,"thread_start_us":17738,"threads_started":1}
I20260809 13:13:00.350274 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.338988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:40953 (local address 127.0.0.1:51150)
0809 13:13:00.342590 (+ 3602us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.342604 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.342718 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.343429 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.343434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.343460 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:00.344047 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.344063 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.345941 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.345947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.347859 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.347873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.349412 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.349418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.349615 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.349644 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.349762 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.349880 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3478,"mutex_wait_us":199,"thread_start_us":3350,"threads_started":1}
I20260809 13:13:00.350297 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.335769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.10.181.129:40953)
0809 13:13:00.342323 (+ 6554us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.342332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.342717 (+ 385us) server_negotiation.cc:407] Connection header received
0809 13:13:00.342845 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.342851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.343090 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.343282 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:00.344239 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.345785 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.348052 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.349210 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.349386 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.349572 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.349810 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.349967 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6384,"thread_start_us":6229,"threads_started":1}
I20260809 13:13:00.349802 3756 tablet_service.cc:1467] Processing CreateTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 (DEFAULT_TABLE table=test-table [id=a7302e77515c4056970987f4714fec58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:00.351586 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b68d08a88604c8b9d829b77e8b8ce69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:00.351351 3564 tablet_service.cc:1467] Processing CreateTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 (DEFAULT_TABLE table=test-table [id=a7302e77515c4056970987f4714fec58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:00.352736 3564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b68d08a88604c8b9d829b77e8b8ce69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:00.353310 3371 tablet_service.cc:1467] Processing CreateTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 (DEFAULT_TABLE table=test-table [id=a7302e77515c4056970987f4714fec58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:00.354516 3371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b68d08a88604c8b9d829b77e8b8ce69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:00.365111 3896 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Bootstrap starting.
I20260809 13:13:00.367127 3899 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Bootstrap starting.
I20260809 13:13:00.368520 3897 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Bootstrap starting.
I20260809 13:13:00.369187 3896 tablet_bootstrap.cc:654] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:00.370509 3899 tablet_bootstrap.cc:654] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:00.370586 3896 log.cc:824] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:00.371387 3897 tablet_bootstrap.cc:654] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:00.371722 3899 log.cc:824] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:00.372577 3897 log.cc:824] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:00.374675 3896 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: No bootstrap required, opened a new log
I20260809 13:13:00.374910 3896 ts_tablet_manager.cc:1397] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 13:13:00.375378 3899 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: No bootstrap required, opened a new log
I20260809 13:13:00.375643 3899 ts_tablet_manager.cc:1397] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:13:00.375723 3897 tablet_bootstrap.cc:492] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: No bootstrap required, opened a new log
I20260809 13:13:00.376006 3897 ts_tablet_manager.cc:1397] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 13:13:00.381759 3896 raft_consensus.cc:348] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.382104 3896 raft_consensus.cc:374] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:00.382203 3896 raft_consensus.cc:729] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cbcb6b8e90247b78cefe486ef96e1e0, State: Initialized, Role: FOLLOWER
I20260809 13:13:00.382752 3896 consensus_queue.cc:260] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.384459 3896 ts_tablet_manager.cc:1428] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260809 13:13:00.385933 3899 raft_consensus.cc:348] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.386092 3897 raft_consensus.cc:348] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.386399 3899 raft_consensus.cc:374] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:00.386534 3899 raft_consensus.cc:729] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d137f235131042d1ae86e584badcf616, State: Initialized, Role: FOLLOWER
I20260809 13:13:00.386552 3897 raft_consensus.cc:374] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:00.386649 3897 raft_consensus.cc:729] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb4b1bd6f748efb6428fa09c842c43, State: Initialized, Role: FOLLOWER
I20260809 13:13:00.387149 3899 consensus_queue.cc:260] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.387264 3897 consensus_queue.cc:260] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.388262 3657 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:00.388916 3899 ts_tablet_manager.cc:1428] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 13:13:00.390183 3858 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:00.390753 3897 ts_tablet_manager.cc:1428] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 13:13:00.405359 3905 raft_consensus.cc:482] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:13:00.405647 3905 raft_consensus.cc:504] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.407167 3905 leader_election.cc:288] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933), 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953)
I20260809 13:13:00.413715 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.407549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:47871 (local address 127.10.181.131:34933)
0809 13:13:00.407765 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.407771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.408037 (+ 266us) server_negotiation.cc:407] Connection header received
0809 13:13:00.408370 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.408376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.408455 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.408519 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:00.409327 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.410122 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.412011 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.412787 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.412932 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.413014 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.413318 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.413474 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:13:00.413882 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.407352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:34933 (local address 127.10.181.130:47871)
0809 13:13:00.407937 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.407956 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.408185 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.408723 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.408727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.408749 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:00.409191 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.409208 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.410253 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.410258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.411845 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.411863 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.412999 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.413005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.413164 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.413192 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.413277 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.413418 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":21,"thread_start_us":280,"threads_started":1}
I20260809 13:13:00.413918 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.408285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:59323 (local address 127.10.181.129:40953)
0809 13:13:00.408482 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.408488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.408684 (+ 196us) server_negotiation.cc:407] Connection header received
0809 13:13:00.408838 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.408842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.408901 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.408973 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:00.409652 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.410386 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.412209 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.412986 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.413142 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.413218 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.413553 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.413684 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:13:00.414067 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.407976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:40953 (local address 127.10.181.130:59323)
0809 13:13:00.408600 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:00.408617 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:00.408730 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:00.409019 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:00.409023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:00.409049 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:00.409489 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.409503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.410528 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.410534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.412049 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:00.412064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.413200 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.413207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:00.413377 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.413401 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:13:00.413513 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 13:13:00.413657 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":20,"thread_start_us":297,"threads_started":1}
I20260809 13:13:00.415083 3791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b68d08a88604c8b9d829b77e8b8ce69" candidate_uuid: "d137f235131042d1ae86e584badcf616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" is_pre_election: true
I20260809 13:13:00.415095 3404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b68d08a88604c8b9d829b77e8b8ce69" candidate_uuid: "d137f235131042d1ae86e584badcf616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" is_pre_election: true
I20260809 13:13:00.415697 3791 raft_consensus.cc:2454] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d137f235131042d1ae86e584badcf616 in term 0.
I20260809 13:13:00.415697 3404 raft_consensus.cc:2454] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d137f235131042d1ae86e584badcf616 in term 0.
I20260809 13:13:00.416392 3492 leader_election.cc:302] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbcb6b8e90247b78cefe486ef96e1e0, d137f235131042d1ae86e584badcf616; no voters:
I20260809 13:13:00.416749 3905 raft_consensus.cc:2781] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:13:00.416894 3905 raft_consensus.cc:482] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:13:00.416999 3905 raft_consensus.cc:3037] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:00.419261 3905 raft_consensus.cc:504] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.419889 3905 leader_election.cc:288] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [CANDIDATE]: Term 1 election: Requested vote from peers ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933), 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953)
I20260809 13:13:00.420363 3791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b68d08a88604c8b9d829b77e8b8ce69" candidate_uuid: "d137f235131042d1ae86e584badcf616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
I20260809 13:13:00.420470 3404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b68d08a88604c8b9d829b77e8b8ce69" candidate_uuid: "d137f235131042d1ae86e584badcf616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
I20260809 13:13:00.420589 3791 raft_consensus.cc:3037] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:00.420682 3404 raft_consensus.cc:3037] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:00.423866 3791 raft_consensus.cc:2454] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d137f235131042d1ae86e584badcf616 in term 1.
I20260809 13:13:00.423866 3404 raft_consensus.cc:2454] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d137f235131042d1ae86e584badcf616 in term 1.
I20260809 13:13:00.424285 3492 leader_election.cc:302] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d137f235131042d1ae86e584badcf616, ddfb4b1bd6f748efb6428fa09c842c43; no voters:
I20260809 13:13:00.424552 3905 raft_consensus.cc:2781] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:13:00.424976 3905 raft_consensus.cc:686] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 1 LEADER]: Becoming Leader. State: Replica: d137f235131042d1ae86e584badcf616, State: Running, Role: LEADER
I20260809 13:13:00.425537 3905 consensus_queue.cc:237] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } }
I20260809 13:13:00.430063 3184 catalog_manager.cc:5495] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 reported cstate change: term changed from 0 to 1, leader changed from <none> to d137f235131042d1ae86e584badcf616 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "d137f235131042d1ae86e584badcf616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } health_report { overall_health: UNKNOWN } } }
W20260809 13:13:00.433004 3464 tablet.cc:2369] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:13:00.457609 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.452646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.10.181.130:33033)
0809 13:13:00.452819 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.452824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.452841 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:13:00.452997 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.453002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.453136 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.453260 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:13:00.453683 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.454426 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.455745 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.456131 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.456217 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.456606 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:13:00.457103 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:13:00.457214 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.457272 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.457336 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:13:00.465660 3404 raft_consensus.cc:1261] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Refusing update from remote peer d137f235131042d1ae86e584badcf616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:13:00.465660 3791 raft_consensus.cc:1261] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Refusing update from remote peer d137f235131042d1ae86e584badcf616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:13:00.466591 3909 consensus_queue.cc:1035] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:13:00.466990 3905 consensus_queue.cc:1035] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:13:00.478812 3658 tablet.cc:2369] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:13:00.493592 3184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:13:00.493848 3184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:13:00.497561 3184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Sending DeleteTablet for 3 replicas of tablet 0b68d08a88604c8b9d829b77e8b8ce69
I20260809 13:13:00.498797 3564 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:00 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33494
I20260809 13:13:00.498983 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 3276
I20260809 13:13:00.499070 3756 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:00 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34726
I20260809 13:13:00.499590 3371 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b68d08a88604c8b9d829b77e8b8ce69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:00 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:51150
I20260809 13:13:00.500010 3932 tablet_replica.cc:330] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: stopping tablet replica
I20260809 13:13:00.500452 3932 raft_consensus.cc:2229] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:13:00.501147 3932 raft_consensus.cc:2258] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:00.503176 3932 ts_tablet_manager.cc:1905] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:13:00.510743 3932 ts_tablet_manager.cc:1918] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:00.511047 3932 log.cc:1196] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/0b68d08a88604c8b9d829b77e8b8ce69
I20260809 13:13:00.511679 3932 ts_tablet_manager.cc:1939] T 0b68d08a88604c8b9d829b77e8b8ce69 P d137f235131042d1ae86e584badcf616: Deleting consensus metadata
I20260809 13:13:00.513666 3165 catalog_manager.cc:4841] TS d137f235131042d1ae86e584badcf616 (127.10.181.130:33033): tablet 0b68d08a88604c8b9d829b77e8b8ce69 (table test-table [id=a7302e77515c4056970987f4714fec58]) successfully deleted
I20260809 13:13:00.533236 3938 tablet_replica.cc:330] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: stopping tablet replica
I20260809 13:13:00.533974 3938 raft_consensus.cc:2229] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:13:00.534708 3938 raft_consensus.cc:2258] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:00.537113 3938 ts_tablet_manager.cc:1905] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:13:00.537748 3933 tablet_replica.cc:330] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: stopping tablet replica
I20260809 13:13:00.538954 3933 raft_consensus.cc:2229] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:13:00.539598 3933 raft_consensus.cc:2258] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:00.541746 3933 ts_tablet_manager.cc:1905] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 13:13:00 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:13:00.544296 3938 ts_tablet_manager.cc:1918] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:00.544449 3938 log.cc:1196] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/0b68d08a88604c8b9d829b77e8b8ce69
I20260809 13:13:00.545017 3938 ts_tablet_manager.cc:1939] T 0b68d08a88604c8b9d829b77e8b8ce69 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting consensus metadata
I20260809 13:13:00.547171 3168 catalog_manager.cc:4841] TS 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953): tablet 0b68d08a88604c8b9d829b77e8b8ce69 (table test-table [id=a7302e77515c4056970987f4714fec58]) successfully deleted
I20260809 13:13:00.548590 3933 ts_tablet_manager.cc:1918] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:00.548755 3933 log.cc:1196] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/0b68d08a88604c8b9d829b77e8b8ce69
I20260809 13:13:00.549269 3933 ts_tablet_manager.cc:1939] T 0b68d08a88604c8b9d829b77e8b8ce69 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting consensus metadata
I20260809 13:13:00.551131 3168 catalog_manager.cc:4841] TS ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933): tablet 0b68d08a88604c8b9d829b77e8b8ce69 (table test-table [id=a7302e77515c4056970987f4714fec58]) successfully deleted
I20260809 13:13:00.552232 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.500857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.10.181.129:40953)
0809 13:13:00.501038 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.501044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.532286 (+ 31242us) server_negotiation.cc:407] Connection header received
0809 13:13:00.532509 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.532515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.532623 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.532721 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:00.535250 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.536390 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.538383 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.539041 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.539170 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.545996 (+ 6826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:00.546008 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:00.546025 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:00.546070 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:00.549313 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.550090 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.550097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.550099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.550232 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.550683 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.550688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.550691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.551127 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:00.551423 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.551774 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.551936 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:13:00.553118 3458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:00.642788 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 3276
I20260809 13:13:00.668354 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 3470
Aug 09 13:13:00 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:13:00.685688 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.669138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.10.181.130:33033)
0809 13:13:00.669330 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.669336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.669363 (+ 27us) server_negotiation.cc:407] Connection header received
0809 13:13:00.669544 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.669554 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.669653 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.669770 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:00.671137 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.672408 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.673843 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.674521 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.674656 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.678858 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:00.678871 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:00.678890 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:00.678931 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:00.682645 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.683349 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.683359 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.683363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.683536 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.683984 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.683989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.683993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.684472 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:00.684814 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.685036 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.685259 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:13:00.686522 3651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:00.802013 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 3470
I20260809 13:13:00.826285 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 3666
Aug 09 13:13:00 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:13:00.840791 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:00.827050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.10.181.131:34933)
0809 13:13:00.827244 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:00.827250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:00.827271 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:13:00.827457 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:00.827462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:00.827553 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:00.827640 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:00.828898 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.829689 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.830996 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:00.831391 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:00.831501 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:00.835914 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:00.835929 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:00.835947 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:00.835981 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:00.838495 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.839110 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.839118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.839120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.839262 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:00.839579 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:00.839582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:00.839584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:00.839938 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:00.840158 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:00.840382 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 13:13:00.840553 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:13:00.841544 3851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:00.928511 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 3666
I20260809 13:13:00.951267 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 3139
I20260809 13:13:00.952416 3232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:01.062001 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 3139
I20260809 13:13:21.092788 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--webserver_interface=127.10.181.190
--webserver_port=38427
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:13:21.446533 3945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:21.446811 3945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:21.446938 3945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:21.453987 3945 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:13:21.454092 3945 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:13:21.454157 3945 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:13:21.454233 3945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:13:21.454306 3945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:21.454399 3945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:21.454458 3945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:13:21.454540 3945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:13:21.464453 3945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=38427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:21.465742 3945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:21.467079 3945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:21.491657 3957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:21.491714 3954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:21.491657 3955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:21.492040 3945 server_base.cc:1034] running on GCE node
I20260809 13:13:21.493032 3945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:21.494604 3945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:21.495843 3945 hybrid_clock.cc:648] HybridClock initialized: now 1786281201495775 us; error 33 us; skew 500 ppm
I20260809 13:13:21.496229 3945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:21 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281201, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:21.499759 3945 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:13:21.501271 3945 webserver.cc:460] Webserver started at http://127.10.181.190:38427/ using document root <none> and password file <none>
I20260809 13:13:21.502074 3945 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:21.502211 3945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:21.508103 3945 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:13:21.511047 3966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:21.512392 3945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:13:21.512563 3945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "739eb820ecf741eab6261d0f16815c58"
format_stamp: "Formatted at 2026-08-09 13:12:58 on dist-test-slave-42z9"
server_key: "669b3bb157e30ed433076903b2290b30"
server_key_iv: "ea18b7c3b467071bb63d0869fffe1584"
server_key_version: "encryptionkey@0"
I20260809 13:13:21.513615 3945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:21.532200 3945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:21.536403 3945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:21.536686 3945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:21.554311 3945 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46013
I20260809 13:13:21.554347 4055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46013 every 8 connection(s)
I20260809 13:13:21.555727 3945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:13:21.562479 4056 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:13:21.563534 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 3945
I20260809 13:13:21.564592 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:40953
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=33139
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:13:21.566835 4056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap starting.
I20260809 13:13:21.588519 4056 log.cc:824] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:21.600890 4056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:13:21.601423 4056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap complete.
I20260809 13:13:21.610524 4056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:21.611308 4056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Initialized, Role: FOLLOWER
I20260809 13:13:21.611946 4056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:21.612164 4056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:13:21.612277 4056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:13:21.612443 4056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 1 FOLLOWER]: Advancing to term 2
I20260809 13:13:21.616184 4056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:21.616725 4056 leader_election.cc:302] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 739eb820ecf741eab6261d0f16815c58; no voters:
I20260809 13:13:21.617316 4056 leader_election.cc:288] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 13:13:21.617523 4063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 13:13:21.619462 4056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:13:21.621737 4063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 LEADER]: Becoming Leader. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Running, Role: LEADER
I20260809 13:13:21.622500 4063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:21.625293 4064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:13:21.625324 4065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 739eb820ecf741eab6261d0f16815c58. Latest consensus state: current_term: 2 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:13:21.625756 4064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:13:21.625761 4065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:13:21.626714 4077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:13:21.645607 4077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:13:21.647162 4077 catalog_manager.cc:1259] Loaded cluster ID: 46d4f6ceb891452298639642c4373e21
I20260809 13:13:21.647330 4077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:13:21.651752 4077 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:13:21.653628 4077 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Loaded TSK: 0
I20260809 13:13:21.682369 4077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Generated new TSK 1
I20260809 13:13:21.682721 4077 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 13:13:21.936035 4058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:21.936338 4058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:21.936491 4058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:21.943583 4058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:21.943689 4058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:21.943848 4058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:13:21.953083 4058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:40953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=33139
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:21.954386 4058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:21.955685 4058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:21.969955 4100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:21.970036 4105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:21.970580 4099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:21.972162 4058 server_base.cc:1034] running on GCE node
I20260809 13:13:21.972724 4058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:21.973636 4058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:21.974834 4058 hybrid_clock.cc:648] HybridClock initialized: now 1786281201974786 us; error 40 us; skew 500 ppm
I20260809 13:13:21.975121 4058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:21 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281201, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:21.978085 4058 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:13:21.979506 4058 webserver.cc:460] Webserver started at http://127.10.181.129:33139/ using document root <none> and password file <none>
I20260809 13:13:21.980160 4058 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:21.980279 4058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:21.985647 4058 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:13:21.988391 4113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:21.989796 4058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:13:21.989941 4058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "1340000bd0c0123d250df58848144eb2"
server_key_iv: "7760292ad755e60663d558721ae87e1c"
server_key_version: "encryptionkey@0"
I20260809 13:13:21.990869 4058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:22.016098 4058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:22.020362 4058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:22.020663 4058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:22.021682 4058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:22.023000 4058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:22.023095 4058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.023211 4058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:22.023280 4058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.053313 4058 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:40953
I20260809 13:13:22.053378 4279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:40953 every 8 connection(s)
I20260809 13:13:22.054775 4058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:13:22.055481 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 4058
I20260809 13:13:22.056275 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:33033
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=42111
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:13:22 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281201, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:13:22.079253 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.057922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:40763)
0809 13:13:22.058861 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:22.058914 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:22.061046 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:22.062437 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:22.062447 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:22.063167 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:13:22.063926 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.063950 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.065536 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.065542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.067306 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.067323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.068288 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.068294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.068555 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.069249 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:13:22.069278 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:13:22.071605 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:13:22.075532 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.075538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.075549 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.075967 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.076440 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.076443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.076445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.076611 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.077427 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:13:22.077432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:13:22.077751 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 13:13:22.077964 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 13:13:22.078316 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":427,"threads_started":1}
I20260809 13:13:22.079728 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.057822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:40763 (local address 127.10.181.190:46013)
0809 13:13:22.058490 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.058506 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.059054 (+ 548us) server_negotiation.cc:407] Connection header received
0809 13:13:22.061627 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.061641 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.061945 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.062232 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:22.064112 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.065336 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.067433 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.068056 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.068191 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.071888 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:22.071898 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:22.071916 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:22.071971 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:22.075403 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.076121 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.076130 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.076138 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.076345 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.076733 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.076737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.076740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.077308 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:22.077532 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:22.078235 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 13:13:22.078627 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":259,"threads_started":1}
I20260809 13:13:22.082433 4280 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:13:22.082755 4280 heartbeater.cc:464] Registering TS with master...
I20260809 13:13:22.083611 4280 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:13:22.086210 3998 ts_manager.cc:194] Registered new tserver with Master: 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953)
I20260809 13:13:22.088301 3998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:40763
W20260809 13:13:22.391207 4290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:22.391443 4290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:22.391554 4290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:22.398406 4290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:22.398499 4290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:22.398610 4290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:13:22.407536 4290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:33033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=42111
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:22.408638 4290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:22.409832 4290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:22.422063 4298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:22.422143 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:22.422113 4300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:22.423357 4290 server_base.cc:1034] running on GCE node
I20260809 13:13:22.423918 4290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:22.424764 4290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:22.425961 4290 hybrid_clock.cc:648] HybridClock initialized: now 1786281202425901 us; error 49 us; skew 500 ppm
I20260809 13:13:22.426259 4290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:22 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281202, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:22.429180 4290 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:13:22.430461 4290 webserver.cc:460] Webserver started at http://127.10.181.130:42111/ using document root <none> and password file <none>
I20260809 13:13:22.431069 4290 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:22.431195 4290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:22.436388 4290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 13:13:22.439033 4308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.440272 4290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:13:22.440412 4290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "d137f235131042d1ae86e584badcf616"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "3ff2892140d26ba74b833e887746ed35"
server_key_iv: "d52481948d8c379c5e376beafa774cea"
server_key_version: "encryptionkey@0"
I20260809 13:13:22.441301 4290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:22.471009 4290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:22.474571 4290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:22.474812 4290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:22.475694 4290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:22.476948 4290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:22.477046 4290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.477147 4290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:22.477221 4290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.507277 4290 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33033
I20260809 13:13:22.507351 4484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33033 every 8 connection(s)
I20260809 13:13:22.508664 4290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:13:22.510072 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 4290
I20260809 13:13:22.510689 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:34933
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=44301
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 13:13:22 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281202, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:13:22.528326 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.511204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:54905 (local address 127.10.181.190:46013)
0809 13:13:22.511378 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.511383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.512734 (+ 1351us) server_negotiation.cc:407] Connection header received
0809 13:13:22.514616 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.514623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.514712 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.514803 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:22.516309 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.517082 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.518796 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.519235 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.519366 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.522931 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:22.522936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:22.522939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:22.522974 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:22.525283 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.526129 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.526134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.526136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.526257 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.526730 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.526733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.526734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.526987 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:22.527098 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:22.527889 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 13:13:22.528081 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:13:22.529529 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.511824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:54905)
0809 13:13:22.512568 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:22.512605 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:22.514336 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:22.515050 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:22.515058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:22.515567 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:13:22.516129 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.516148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.517240 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.517244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.518651 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.518664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.519496 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.519506 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.519738 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.520357 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:13:22.520380 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:13:22.522705 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:13:22.525428 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.525464 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.525480 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.526003 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.526374 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.526380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.526383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.526639 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.527106 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:13:22.527113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:13:22.527499 (+ 386us) client_negotiation.cc:769] Sending connection context
0809 13:13:22.527803 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 13:13:22.528278 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":296,"threads_started":1}
I20260809 13:13:22.530941 4485 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:13:22.531243 4485 heartbeater.cc:464] Registering TS with master...
I20260809 13:13:22.531920 4485 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:13:22.533382 3998 ts_manager.cc:194] Registered new tserver with Master: d137f235131042d1ae86e584badcf616 (127.10.181.130:33033)
I20260809 13:13:22.534389 3998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:54905
W20260809 13:13:22.845711 4491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:22.845947 4491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:22.846053 4491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:22.852839 4491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:22.852927 4491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:22.853044 4491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:13:22.862195 4491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:34933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=44301
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:22.863242 4491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:22.864434 4491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:22.876513 4497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:22.876477 4498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:22.876541 4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:22.877532 4491 server_base.cc:1034] running on GCE node
I20260809 13:13:22.878089 4491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:22.878882 4491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:22.880075 4491 hybrid_clock.cc:648] HybridClock initialized: now 1786281202880030 us; error 40 us; skew 500 ppm
I20260809 13:13:22.880367 4491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:22 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281202, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:22.883263 4491 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:13:22.884673 4491 webserver.cc:460] Webserver started at http://127.10.181.131:44301/ using document root <none> and password file <none>
I20260809 13:13:22.885295 4491 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:22.885424 4491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:22.890388 4491 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 13:13:22.892920 4509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.894199 4491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:13:22.894340 4491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
format_stamp: "Formatted at 2026-08-09 13:13:00 on dist-test-slave-42z9"
server_key: "409b0ae1297dabe0df2ceaded0653255"
server_key_iv: "bdf03c78645ff3092d0934cf65e3048f"
server_key_version: "encryptionkey@0"
I20260809 13:13:22.895277 4491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:22.913086 4491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:22.916533 4491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:22.916764 4491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:22.917691 4491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:22.918937 4491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:22.919040 4491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.919154 4491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:22.919239 4491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:22.948760 4491 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:34933
I20260809 13:13:22.948818 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:34933 every 8 connection(s)
I20260809 13:13:22.950124 4491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:13:22.951509 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 4491
Aug 09 13:13:22 dist-test-slave-42z9 krb5kdc[3122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281202, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:13:22.968287 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.952419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.10.181.190:46013)
0809 13:13:22.952918 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.952921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.954965 (+ 2044us) server_negotiation.cc:407] Connection header received
0809 13:13:22.955194 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.955198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.955265 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.955358 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:22.956930 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.957746 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.959119 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.959542 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.959680 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.962507 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:22.962512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:22.962515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:22.962545 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:22.965639 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.966328 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.966334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.966337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.966477 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.966826 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.966830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.966832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.967102 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:22.967357 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:22.967692 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 13:13:22.967919 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":424,"thread_start_us":242,"threads_started":1}
I20260809 13:13:22.969652 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.952413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:60623 (local address 127.10.181.190:46013)
0809 13:13:22.952595 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.952602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.954042 (+ 1440us) server_negotiation.cc:407] Connection header received
0809 13:13:22.955877 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.955883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.955956 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.956039 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:22.957458 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.958221 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.959696 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.960127 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.960230 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.963644 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:22.963650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:22.963655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:22.963703 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:22.966629 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.967265 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.967270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.967274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.967399 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.967793 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.967798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.967802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.968071 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:22.968219 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:22.969096 (+ 877us) server_negotiation.cc:299] Negotiation successful
0809 13:13:22.969295 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:13:22.970212 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.953043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:60623)
0809 13:13:22.953894 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:22.953937 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:22.955627 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:22.956276 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:22.956284 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:22.956754 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:13:22.957271 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.957288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.958382 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.958386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.959555 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:22.959568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.960336 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.960344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:22.960558 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.961115 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:13:22.961137 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:13:22.963424 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:13:22.966744 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.966752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.966763 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.967154 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.967500 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:13:22.967505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:13:22.967508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:13:22.967685 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:13:22.968223 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:13:22.968232 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:13:22.968591 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 13:13:22.968811 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 13:13:22.969196 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":357,"threads_started":1}
I20260809 13:13:22.971464 4673 heartbeater.cc:347] Connected to a master server at 127.10.181.190:46013
I20260809 13:13:22.971724 4673 heartbeater.cc:464] Registering TS with master...
I20260809 13:13:22.972348 4673 heartbeater.cc:511] Master 127.10.181.190:46013 requested a full tablet report, sending...
I20260809 13:13:22.973711 3997 ts_manager.cc:194] Registered new tserver with Master: ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933)
I20260809 13:13:22.974542 3997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:60623
I20260809 13:13:22.983011 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:13:22.988751 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.983824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.10.181.190:46013)
0809 13:13:22.983985 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.983990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.984427 (+ 437us) server_negotiation.cc:407] Connection header received
0809 13:13:22.984585 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.984588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.984703 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.984810 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:13:22.985176 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.985941 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.987242 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.987613 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.987697 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.988096 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:13:22.988340 (+ 244us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 13:13:22.988598 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44434: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 13:13:22.988862 4285 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44434: token expired
W20260809 13:13:22.989513 3870 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.181.190:46013)
I20260809 13:13:23.001157 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:22.990042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.10.181.190:46013)
0809 13:13:22.990214 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:22.990219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:22.990235 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:13:22.990273 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:22.990276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:22.990336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:22.990411 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:22.991655 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.992332 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.993774 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:22.994229 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:22.994378 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:22.996770 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:22.996775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:22.996778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:22.996807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:22.999023 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:22.999658 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:22.999662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:22.999664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:22.999801 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.000078 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.000081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.000083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.000356 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:23.000528 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.000738 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.000883 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:13:23.004163 3997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:13:23.011165 3997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:13:23.046834 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.022384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.10.181.129:40953)
0809 13:13:23.036680 (+ 14296us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.036689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.037612 (+ 923us) server_negotiation.cc:407] Connection header received
0809 13:13:23.037748 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.037755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.037929 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.038085 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:23.038980 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.040671 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.044124 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.045407 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.045603 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.045762 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.046319 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.046491 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14108,"thread_start_us":13932,"threads_started":1}
I20260809 13:13:23.047115 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.023027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.10.181.130:33033)
0809 13:13:23.035185 (+ 12158us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.035197 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.036096 (+ 899us) server_negotiation.cc:407] Connection header received
0809 13:13:23.036510 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.036518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.036772 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.036902 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:23.037803 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.040666 (+ 2863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.043986 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.045971 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.046137 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.046303 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.046663 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.046825 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11941,"thread_start_us":12162,"threads_started":1}
I20260809 13:13:23.047565 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.022389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33033 (local address 127.0.0.1:40808)
0809 13:13:23.035927 (+ 13538us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:23.035963 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:23.036245 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:23.037154 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:23.037160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:23.037187 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:23.037603 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.037617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.040873 (+ 3256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.040879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.043812 (+ 2933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.043828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.046230 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.046238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.046458 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.046490 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:13:23.046581 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 13:13:23.046773 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13303,"mutex_wait_us":4,"thread_start_us":13174,"threads_started":1}
I20260809 13:13:23.047592 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.022400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:40953 (local address 127.0.0.1:56458)
0809 13:13:23.037491 (+ 15091us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:23.037508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:23.037620 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:23.038350 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:23.038354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:23.038379 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:23.038770 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.038784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.040869 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.040875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.043988 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.043998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.045913 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.045917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.046055 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.046104 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 13:13:23.047049 (+ 945us) client_negotiation.cc:240] Negotiation successful
0809 13:13:23.047176 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14950,"thread_start_us":15276,"threads_started":1}
I20260809 13:13:23.050385 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.038136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:34933 (local address 127.0.0.1:51836)
0809 13:13:23.042355 (+ 4219us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:23.042370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:23.042490 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:23.042988 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:23.042992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:23.043016 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:23.043433 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.043446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.045244 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.045249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.047235 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.047251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.049627 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.049634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.049789 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.049821 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:13:23.049916 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 13:13:23.050029 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4116,"mutex_wait_us":1,"thread_start_us":3977,"threads_started":1}
I20260809 13:13:23.050664 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.038493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.10.181.131:34933)
0809 13:13:23.041668 (+ 3175us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.041674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.042476 (+ 802us) server_negotiation.cc:407] Connection header received
0809 13:13:23.042650 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.042656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.042816 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.042971 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:23.043605 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.045072 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.047433 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.048437 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.048582 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.048696 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.049983 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.050143 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3023,"thread_start_us":2865,"threads_started":1}
I20260809 13:13:23.051260 4389 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 (DEFAULT_TABLE table=test-table [id=5e5f601e10f6421ba78fc510399f4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:23.051879 4184 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 (DEFAULT_TABLE table=test-table [id=5e5f601e10f6421ba78fc510399f4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:23.052698 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea8a77980f4f17937dbc22e1cc11c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:23.053249 4184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea8a77980f4f17937dbc22e1cc11c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:23.057004 4574 tablet_service.cc:1467] Processing CreateTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 (DEFAULT_TABLE table=test-table [id=5e5f601e10f6421ba78fc510399f4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:13:23.058377 4574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8ea8a77980f4f17937dbc22e1cc11c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:13:23.068001 4699 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Bootstrap starting.
I20260809 13:13:23.070026 4701 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Bootstrap starting.
I20260809 13:13:23.072518 4699 tablet_bootstrap.cc:654] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:23.073746 4699 log.cc:824] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:23.076200 4701 tablet_bootstrap.cc:654] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:23.076517 4699 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: No bootstrap required, opened a new log
I20260809 13:13:23.076769 4699 ts_tablet_manager.cc:1397] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 13:13:23.077219 4701 log.cc:824] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:23.079698 4701 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: No bootstrap required, opened a new log
I20260809 13:13:23.079921 4701 ts_tablet_manager.cc:1397] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 13:13:23.083451 4702 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Bootstrap starting.
I20260809 13:13:23.084182 4699 raft_consensus.cc:348] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.084636 4699 raft_consensus.cc:374] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:23.084743 4699 raft_consensus.cc:729] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cbcb6b8e90247b78cefe486ef96e1e0, State: Initialized, Role: FOLLOWER
I20260809 13:13:23.085319 4699 consensus_queue.cc:260] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.086413 4280 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:23.086922 4702 tablet_bootstrap.cc:654] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Neither blocks nor log segments found. Creating new log.
I20260809 13:13:23.088126 4702 log.cc:824] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:23.088248 4699 ts_tablet_manager.cc:1428] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 13:13:23.088423 4701 raft_consensus.cc:348] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.088872 4701 raft_consensus.cc:374] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:23.088970 4701 raft_consensus.cc:729] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d137f235131042d1ae86e584badcf616, State: Initialized, Role: FOLLOWER
I20260809 13:13:23.089617 4701 consensus_queue.cc:260] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.091243 4485 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:23.091281 4701 ts_tablet_manager.cc:1428] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 13:13:23.091796 4702 tablet_bootstrap.cc:492] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: No bootstrap required, opened a new log
I20260809 13:13:23.092033 4702 ts_tablet_manager.cc:1397] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260809 13:13:23.099460 4702 raft_consensus.cc:348] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.099889 4702 raft_consensus.cc:374] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:13:23.100003 4702 raft_consensus.cc:729] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb4b1bd6f748efb6428fa09c842c43, State: Initialized, Role: FOLLOWER
I20260809 13:13:23.100545 4702 consensus_queue.cc:260] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.101475 4673 heartbeater.cc:503] Master 127.10.181.190:46013 was elected leader, sending a full tablet report...
I20260809 13:13:23.101866 4702 ts_tablet_manager.cc:1428] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 13:13:23.139936 4710 raft_consensus.cc:482] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:13:23.140254 4710 raft_consensus.cc:504] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.142072 4710 leader_election.cc:288] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953), d137f235131042d1ae86e584badcf616 (127.10.181.130:33033)
I20260809 13:13:23.148108 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.142306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:40953 (local address 127.10.181.131:40599)
0809 13:13:23.142610 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:23.142631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:23.142781 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:23.143372 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:23.143375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:23.143402 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:23.143587 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.143597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.144707 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.144712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.146364 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.146376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.147432 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.147438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.147609 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.147635 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 13:13:23.147714 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 13:13:23.147836 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 13:13:23.148123 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.142417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:40599 (local address 127.10.181.129:40953)
0809 13:13:23.142641 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.142647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.142917 (+ 270us) server_negotiation.cc:407] Connection header received
0809 13:13:23.143006 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.143011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.143091 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.143191 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:23.143767 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.144578 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.146518 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.147282 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.147418 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.147492 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.147755 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.147883 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:13:23.148964 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.142830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:59611 (local address 127.10.181.130:33033)
0809 13:13:23.143035 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.143041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.143519 (+ 478us) server_negotiation.cc:407] Connection header received
0809 13:13:23.143751 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.143755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.143827 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.143892 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:13:23.144542 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.145653 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.147297 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.148050 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.148204 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.148273 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.148582 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.148720 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:13:23.148991 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.142560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33033 (local address 127.10.181.131:59611)
0809 13:13:23.143402 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:23.143420 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:13:23.143562 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:13:23.144097 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:13:23.144100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:13:23.144120 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:13:23.144382 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.144392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.145814 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.145818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.147140 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:13:23.147152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.148250 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.148256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:13:23.148414 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.148452 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 13:13:23.148549 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:13:23.148665 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":487,"threads_started":1}
I20260809 13:13:23.149125 4215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea8a77980f4f17937dbc22e1cc11c0" candidate_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" is_pre_election: true
I20260809 13:13:23.149722 4215 raft_consensus.cc:2454] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfb4b1bd6f748efb6428fa09c842c43 in term 0.
I20260809 13:13:23.149932 4411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea8a77980f4f17937dbc22e1cc11c0" candidate_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d137f235131042d1ae86e584badcf616" is_pre_election: true
I20260809 13:13:23.150425 4512 leader_election.cc:302] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbcb6b8e90247b78cefe486ef96e1e0, ddfb4b1bd6f748efb6428fa09c842c43; no voters:
I20260809 13:13:23.150498 4411 raft_consensus.cc:2454] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfb4b1bd6f748efb6428fa09c842c43 in term 0.
I20260809 13:13:23.151005 4710 raft_consensus.cc:2781] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:13:23.151175 4710 raft_consensus.cc:482] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:13:23.151292 4710 raft_consensus.cc:3037] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:23.154546 4710 raft_consensus.cc:504] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.155331 4710 leader_election.cc:288] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [CANDIDATE]: Term 1 election: Requested vote from peers 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953), d137f235131042d1ae86e584badcf616 (127.10.181.130:33033)
I20260809 13:13:23.155548 4215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea8a77980f4f17937dbc22e1cc11c0" candidate_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
I20260809 13:13:23.155771 4215 raft_consensus.cc:3037] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:23.155776 4411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8ea8a77980f4f17937dbc22e1cc11c0" candidate_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d137f235131042d1ae86e584badcf616"
I20260809 13:13:23.155982 4411 raft_consensus.cc:3037] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:13:23.158951 4215 raft_consensus.cc:2454] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfb4b1bd6f748efb6428fa09c842c43 in term 1.
I20260809 13:13:23.159238 4411 raft_consensus.cc:2454] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfb4b1bd6f748efb6428fa09c842c43 in term 1.
I20260809 13:13:23.159359 4512 leader_election.cc:302] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbcb6b8e90247b78cefe486ef96e1e0, ddfb4b1bd6f748efb6428fa09c842c43; no voters:
I20260809 13:13:23.159744 4710 raft_consensus.cc:2781] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:13:23.160153 4710 raft_consensus.cc:686] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 LEADER]: Becoming Leader. State: Replica: ddfb4b1bd6f748efb6428fa09c842c43, State: Running, Role: LEADER
I20260809 13:13:23.160597 4710 consensus_queue.cc:237] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } }
I20260809 13:13:23.165319 3997 catalog_manager.cc:5495] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfb4b1bd6f748efb6428fa09c842c43" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 34933 } health_report { overall_health: HEALTHY } } }
I20260809 13:13:23.184381 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.179628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.10.181.131:34933)
0809 13:13:23.179794 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.179799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.179817 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:13:23.179922 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.179926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.179999 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.180114 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:13:23.180416 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.181135 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.182438 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.182828 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.182917 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.183430 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:13:23.183928 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:13:23.184027 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.184090 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.184151 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 13:13:23.192081 4411 raft_consensus.cc:1261] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Refusing update from remote peer ddfb4b1bd6f748efb6428fa09c842c43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:13:23.192101 4215 raft_consensus.cc:1261] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Refusing update from remote peer ddfb4b1bd6f748efb6428fa09c842c43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:13:23.193028 4712 consensus_queue.cc:1035] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d137f235131042d1ae86e584badcf616" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:13:23.193346 4710 consensus_queue.cc:1035] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cbcb6b8e90247b78cefe486ef96e1e0" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 40953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:13:23.201557 4676 tablet.cc:2369] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:13:23.205294 4717 mvcc.cc:205] Tried to move back new op lower bound from 7316607808265711616 to 7316607808149483520. Current Snapshot: MvccSnapshot[applied={T|T < 7316607808265711616}]
I20260809 13:13:23.216892 3997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:13:23.217170 3997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:13:23.220695 3997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Sending DeleteTablet for 3 replicas of tablet b8ea8a77980f4f17937dbc22e1cc11c0
I20260809 13:13:23.221729 4184 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:56458
I20260809 13:13:23.221917 4389 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:40808
I20260809 13:13:23.222584 4733 tablet_replica.cc:330] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: stopping tablet replica
I20260809 13:13:23.222905 4574 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ea8a77980f4f17937dbc22e1cc11c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:13:23 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:51836
I20260809 13:13:23.223142 4733 raft_consensus.cc:2229] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:13:23.223778 4733 raft_consensus.cc:2258] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:23.223949 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 4058
I20260809 13:13:23.224596 4735 tablet_replica.cc:330] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: stopping tablet replica
I20260809 13:13:23.225018 4735 raft_consensus.cc:2229] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:13:23.226257 4734 tablet_replica.cc:330] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: stopping tablet replica
I20260809 13:13:23.226626 4734 raft_consensus.cc:2229] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:13:23.227020 4734 raft_consensus.cc:2258] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:23.227074 4735 raft_consensus.cc:2258] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:13:23.228809 4734 ts_tablet_manager.cc:1905] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:13:23.228988 4735 ts_tablet_manager.cc:1905] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:13:23.229810 4733 ts_tablet_manager.cc:1905] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:13:23.235371 4734 ts_tablet_manager.cc:1918] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:23.235528 4734 log.cc:1196] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/b8ea8a77980f4f17937dbc22e1cc11c0
I20260809 13:13:23.235929 4735 ts_tablet_manager.cc:1918] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:23.236092 4734 ts_tablet_manager.cc:1939] T b8ea8a77980f4f17937dbc22e1cc11c0 P d137f235131042d1ae86e584badcf616: Deleting consensus metadata
I20260809 13:13:23.236160 4735 log.cc:1196] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/b8ea8a77980f4f17937dbc22e1cc11c0
I20260809 13:13:23.236789 4735 ts_tablet_manager.cc:1939] T b8ea8a77980f4f17937dbc22e1cc11c0 P ddfb4b1bd6f748efb6428fa09c842c43: Deleting consensus metadata
I20260809 13:13:23.237936 3969 catalog_manager.cc:4841] TS d137f235131042d1ae86e584badcf616 (127.10.181.130:33033): tablet b8ea8a77980f4f17937dbc22e1cc11c0 (table test-table [id=5e5f601e10f6421ba78fc510399f4598]) successfully deleted
I20260809 13:13:23.238157 4733 ts_tablet_manager.cc:1918] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:13:23.238370 4733 log.cc:1196] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/b8ea8a77980f4f17937dbc22e1cc11c0
I20260809 13:13:23.238410 3970 catalog_manager.cc:4841] TS ddfb4b1bd6f748efb6428fa09c842c43 (127.10.181.131:34933): tablet b8ea8a77980f4f17937dbc22e1cc11c0 (table test-table [id=5e5f601e10f6421ba78fc510399f4598]) successfully deleted
I20260809 13:13:23.238801 4733 ts_tablet_manager.cc:1939] T b8ea8a77980f4f17937dbc22e1cc11c0 P 7cbcb6b8e90247b78cefe486ef96e1e0: Deleting consensus metadata
I20260809 13:13:23.240245 3970 catalog_manager.cc:4841] TS 7cbcb6b8e90247b78cefe486ef96e1e0 (127.10.181.129:40953): tablet b8ea8a77980f4f17937dbc22e1cc11c0 (table test-table [id=5e5f601e10f6421ba78fc510399f4598]) successfully deleted
I20260809 13:13:23.241797 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.225034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.10.181.129:40953)
0809 13:13:23.225201 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.225207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.225226 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:13:23.225285 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.225290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.225379 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.225551 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:23.227280 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.228317 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.230222 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.230813 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.230940 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.234390 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:23.234406 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:23.234423 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:23.234470 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:23.238681 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.239447 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.239457 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.239460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.239634 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.240126 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.240132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.240136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.240590 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:23.240918 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.241154 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.241383 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 13:13:23.242414 4273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:23.350533 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 4058
I20260809 13:13:23.376066 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 4290
I20260809 13:13:23.391158 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.376946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.10.181.130:33033)
0809 13:13:23.377169 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.377176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.377200 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:13:23.377297 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.377303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.377392 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.377675 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:23.378933 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.380149 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.381343 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.381996 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.382112 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.384659 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:23.384681 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:23.384703 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:23.384752 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:23.388405 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.389062 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.389071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.389075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.389232 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.389577 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.389583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.389586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.390064 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:23.390375 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.390547 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.390774 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:13:23.391912 4479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:23.478998 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 4290
I20260809 13:13:23.503310 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 4491
I20260809 13:13:23.519220 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:23.504144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.10.181.131:34933)
0809 13:13:23.504329 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:13:23.504333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:13:23.504355 (+ 22us) server_negotiation.cc:407] Connection header received
0809 13:13:23.504569 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:13:23.504574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:13:23.504638 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:13:23.504715 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:13:23.506572 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.507391 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.509256 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:13:23.509866 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:13:23.509959 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:13:23.513242 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:13:23.513254 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:13:23.513273 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:13:23.513315 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:13:23.516201 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.517085 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.517092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.517094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.517260 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:13:23.517725 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:13:23.517728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:13:23.517730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:13:23.518137 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:13:23.518391 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 13:13:23.518737 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 13:13:23.518899 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 13:13:23.520031 4667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:23.641296 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 4491
I20260809 13:13:23.666289 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 3945
I20260809 13:13:23.667446 4051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:13:23.780742 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 3945
I20260809 13:13:43.809895 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--webserver_interface=127.10.181.190
--webserver_port=38427
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:13:44.143102 4743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:44.143378 4743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:44.143514 4743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:44.150538 4743 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 13:13:44.150636 4743 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 13:13:44.150699 4743 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 13:13:44.150779 4743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:13:44.150851 4743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:44.150941 4743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:44.151000 4743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:13:44.151070 4743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:13:44.160490 4743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=38427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:44.161734 4743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:44.163018 4743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:44.174533 4748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:44.174618 4755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:44.174583 4752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:44.175426 4743 server_base.cc:1034] running on GCE node
I20260809 13:13:44.176205 4743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:44.177595 4743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:44.178820 4743 hybrid_clock.cc:648] HybridClock initialized: now 1786281224178757 us; error 37 us; skew 500 ppm
I20260809 13:13:44.179134 4743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:44 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:44.186547 4743 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:13:44.187940 4743 webserver.cc:460] Webserver started at http://127.10.181.190:38427/ using document root <none> and password file <none>
I20260809 13:13:44.188609 4743 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:44.188710 4743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:44.194003 4743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 13:13:44.196831 4764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:44.198107 4743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:13:44.198283 4743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "739eb820ecf741eab6261d0f16815c58"
format_stamp: "Formatted at 2026-08-09 13:12:58 on dist-test-slave-42z9"
server_key: "669b3bb157e30ed433076903b2290b30"
server_key_iv: "ea18b7c3b467071bb63d0869fffe1584"
server_key_version: "encryptionkey@0"
I20260809 13:13:44.199224 4743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:44.217006 4743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:44.221026 4743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:44.221287 4743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:44.238250 4743 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46013
I20260809 13:13:44.238308 4843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46013 every 8 connection(s)
I20260809 13:13:44.239733 4743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:13:44.246377 4844 sys_catalog.cc:263] Verifying existing consensus state
I20260809 13:13:44.248860 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 4743
I20260809 13:13:44.249583 4844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap starting.
I20260809 13:13:44.249735 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:40953
--local_ip_for_outbound_sockets=127.10.181.129
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=33139
--webserver_interface=127.10.181.129
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:13:44.271163 4844 log.cc:824] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Log is configured to *not* fsync() on all Append() calls
I20260809 13:13:44.292822 4844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 13:13:44.293502 4844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Bootstrap complete.
I20260809 13:13:44.306613 4844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:44.307802 4844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Initialized, Role: FOLLOWER
I20260809 13:13:44.308494 4844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:44.308761 4844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:13:44.308876 4844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:13:44.309059 4844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 2 FOLLOWER]: Advancing to term 3
I20260809 13:13:44.314386 4844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:44.314946 4844 leader_election.cc:302] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 739eb820ecf741eab6261d0f16815c58; no voters:
I20260809 13:13:44.315605 4844 leader_election.cc:288] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 13:13:44.315737 4851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 13:13:44.316825 4851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [term 3 LEADER]: Becoming Leader. State: Replica: 739eb820ecf741eab6261d0f16815c58, State: Running, Role: LEADER
I20260809 13:13:44.317356 4851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } }
I20260809 13:13:44.317741 4844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:13:44.319955 4853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 739eb820ecf741eab6261d0f16815c58. Latest consensus state: current_term: 3 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:13:44.319962 4852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "739eb820ecf741eab6261d0f16815c58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739eb820ecf741eab6261d0f16815c58" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46013 } } }
I20260809 13:13:44.320483 4853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:13:44.320590 4852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58 [sys.catalog]: This master's current role is: LEADER
I20260809 13:13:44.333895 4860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:13:44.341917 4860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:13:44.343402 4860 catalog_manager.cc:1259] Loaded cluster ID: 46d4f6ceb891452298639642c4373e21
I20260809 13:13:44.343551 4860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:13:44.347563 4860 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:13:44.349151 4860 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Loaded TSK: 0
I20260809 13:13:44.349356 4860 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 739eb820ecf741eab6261d0f16815c58: Loaded TSK: 1
F20260809 13:13:44.354498 4860 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f819ee799ad google::LogMessage::Fail() at ??:0
@ 0x7f819ee7c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f819ee794de google::LogMessage::Flush() at ??:0
@ 0x7f819ee7aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f81a2bdd7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f81a2bcc73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f81b48a81c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f81b48a7783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f81b49332a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f81b49330c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f81b497d5e9 std::function<>::operator()() at ??:0
@ 0x7f81b48abcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f81b48a9c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f81b4931610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f81b4931439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f81a1ac5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f81a1ac891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f81a1ac8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f81a1aa3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81a4d7d6db start_thread at ??:0
@ 0x7f819dd1361f clone at ??:0
W20260809 13:13:44.660099 4847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:44.660342 4847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:44.660460 4847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:44.667704 4847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:44.667814 4847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:44.667980 4847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:13:44.677201 4847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:40953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=33139
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:44.678375 4847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:44.679620 4847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:44.692108 4914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:44.692181 4919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:44.692152 4915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:44.693082 4847 server_base.cc:1034] running on GCE node
I20260809 13:13:44.693778 4847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:44.694721 4847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:44.695921 4847 hybrid_clock.cc:648] HybridClock initialized: now 1786281224695867 us; error 44 us; skew 500 ppm
I20260809 13:13:44.696215 4847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:44 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:44.699267 4847 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:13:44.700591 4847 webserver.cc:460] Webserver started at http://127.10.181.129:33139/ using document root <none> and password file <none>
I20260809 13:13:44.701236 4847 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:44.701357 4847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:44.706765 4847 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 13:13:44.709316 4929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:44.710537 4847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 13:13:44.710706 4847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "7cbcb6b8e90247b78cefe486ef96e1e0"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "1340000bd0c0123d250df58848144eb2"
server_key_iv: "7760292ad755e60663d558721ae87e1c"
server_key_version: "encryptionkey@0"
I20260809 13:13:44.711613 4847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:44.730500 4847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:44.734125 4847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:44.734362 4847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:44.735292 4847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:44.736645 4847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:44.736749 4847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:44.736861 4847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:44.736933 4847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:44.767426 4847 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:40953
I20260809 13:13:44.767488 5106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:40953 every 8 connection(s)
I20260809 13:13:44.768810 4847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:13:44.773316 5109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:44.771579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:47645)
0809 13:13:44.772305 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:44.772526 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":274,"threads_started":1}
W20260809 13:13:44.773967 5107 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46013 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:46013: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
I20260809 13:13:44.774735 5109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:44.774416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:36097)
0809 13:13:44.774562 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:44.774635 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:13:44.775488 5109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:44.775192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:38625)
0809 13:13:44.775323 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:44.775390 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 13:13:44.775704 5107 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:13:44.776796 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 4847
I20260809 13:13:44.777427 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:33033
--local_ip_for_outbound_sockets=127.10.181.130
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=42111
--webserver_interface=127.10.181.130
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:13:45.111624 5113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:45.111881 5113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:45.112015 5113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:45.119032 5113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:45.119129 5113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:45.119272 5113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:13:45.128583 5113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:33033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=42111
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:45.129786 5113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:45.131096 5113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:45.143863 5121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:45.143908 5128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:45.144114 5122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:45.145112 5113 server_base.cc:1034] running on GCE node
I20260809 13:13:45.145735 5113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:45.146627 5113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:45.147823 5113 hybrid_clock.cc:648] HybridClock initialized: now 1786281225147775 us; error 39 us; skew 500 ppm
I20260809 13:13:45.148114 5113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:45 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281225, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:45.151178 5113 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:13:45.154212 5113 webserver.cc:460] Webserver started at http://127.10.181.130:42111/ using document root <none> and password file <none>
I20260809 13:13:45.155066 5113 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:45.155210 5113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:45.161177 5113 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 13:13:45.163831 5141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.165057 5113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:13:45.165217 5113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "d137f235131042d1ae86e584badcf616"
format_stamp: "Formatted at 2026-08-09 13:12:59 on dist-test-slave-42z9"
server_key: "3ff2892140d26ba74b833e887746ed35"
server_key_iv: "d52481948d8c379c5e376beafa774cea"
server_key_version: "encryptionkey@0"
I20260809 13:13:45.166227 5113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:45.180207 5113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:45.183698 5113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:45.183934 5113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:45.184849 5113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:45.186168 5113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:45.186278 5113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.186379 5113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:45.186451 5113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.217344 5113 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33033
I20260809 13:13:45.217423 5310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33033 every 8 connection(s)
I20260809 13:13:45.218801 5113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:13:45.223414 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.221626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:48279)
0809 13:13:45.222333 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.222581 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":257,"threads_started":1}
W20260809 13:13:45.224153 5311 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46013 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:46013: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
I20260809 13:13:45.224924 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.224619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:35525)
0809 13:13:45.224754 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.224821 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 13:13:45.225693 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.225386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:45783)
0809 13:13:45.225520 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.225596 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 13:13:45.225914 5311 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:13:45.227269 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 5113
I20260809 13:13:45.227877 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:34933
--local_ip_for_outbound_sockets=127.10.181.131
--tserver_master_addrs=127.10.181.190:46013
--webserver_port=44301
--webserver_interface=127.10.181.131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:13:45.567823 5315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:13:45.568063 5315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:13:45.568204 5315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:13:45.575163 5315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:13:45.575258 5315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:13:45.575402 5315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:13:45.584756 5315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:34933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=44301
--tserver_master_addrs=127.10.181.190:46013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:13:45.586087 5315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:13:45.587405 5315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:13:45.599958 5321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:45.600004 5323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:13:45.600032 5325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:13:45.600703 5315 server_base.cc:1034] running on GCE node
I20260809 13:13:45.601269 5315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:13:45.602149 5315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:13:45.603341 5315 hybrid_clock.cc:648] HybridClock initialized: now 1786281225603286 us; error 53 us; skew 500 ppm
I20260809 13:13:45.603629 5315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:13:45 dist-test-slave-42z9 krb5kdc[3122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281225, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:13:45.606561 5315 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:13:45.607857 5315 webserver.cc:460] Webserver started at http://127.10.181.131:44301/ using document root <none> and password file <none>
I20260809 13:13:45.608469 5315 fs_manager.cc:359] Metadata directory not provided
I20260809 13:13:45.608590 5315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:13:45.613610 5315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:13:45.616237 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.617480 5315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:13:45.617635 5315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "ddfb4b1bd6f748efb6428fa09c842c43"
format_stamp: "Formatted at 2026-08-09 13:13:00 on dist-test-slave-42z9"
server_key: "409b0ae1297dabe0df2ceaded0653255"
server_key_iv: "bdf03c78645ff3092d0934cf65e3048f"
server_key_version: "encryptionkey@0"
I20260809 13:13:45.618539 5315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:13:45.634389 5315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:13:45.638309 5315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:13:45.638546 5315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:13:45.639446 5315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:13:45.640704 5315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:13:45.640806 5315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.640910 5315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:13:45.640983 5315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:13:45.671298 5315 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:34933
I20260809 13:13:45.671362 5509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:34933 every 8 connection(s)
I20260809 13:13:45.672694 5315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:13:45.677153 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.675427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:41619)
0809 13:13:45.676137 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.676363 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":269,"threads_started":1}
I20260809 13:13:45.677820 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 5315
W20260809 13:13:45.677865 5510 heartbeater.cc:644] Failed to heartbeat to 127.10.181.190:46013 (0 consecutive failures): Network error: Failed to ping master at 127.10.181.190:46013: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
I20260809 13:13:45.678627 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.678316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:57723)
0809 13:13:45.678449 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.678514 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 13:13:45.679543 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.679170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:52139)
0809 13:13:45.679317 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.679415 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:13:45.679759 5510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 13:13:45.776996 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:45.776186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:44885)
0809 13:13:45.776623 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:45.776728 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":198,"threads_started":1}
I20260809 13:13:46.227183 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:46.226391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:37341)
0809 13:13:46.226847 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:46.226954 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":222,"threads_started":1}
I20260809 13:13:46.681011 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:46.680238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:35143)
0809 13:13:46.680668 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:46.680765 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":207,"threads_started":1}
I20260809 13:13:46.778600 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:46.777813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:42877)
0809 13:13:46.778298 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:46.778393 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":258,"threads_started":1}
I20260809 13:13:47.228693 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:47.227975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:59321)
0809 13:13:47.228433 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:47.228517 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":221,"threads_started":1}
I20260809 13:13:47.682520 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:47.681843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:51541)
0809 13:13:47.682279 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:47.682359 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":207,"threads_started":1}
I20260809 13:13:47.780084 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:47.779342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:37003)
0809 13:13:47.779837 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:47.779920 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":247,"threads_started":1}
I20260809 13:13:48.230170 5523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:48.229413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:55869)
0809 13:13:48.229922 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:48.230002 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":256,"threads_started":1}
I20260809 13:13:48.684039 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:48.683303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:35849)
0809 13:13:48.683751 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:48.683829 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":246,"threads_started":1}
I20260809 13:13:48.781752 5527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:48.780949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:56775)
0809 13:13:48.781430 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:48.781549 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":244,"threads_started":1}
I20260809 13:13:49.231735 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:49.231008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:59067)
0809 13:13:49.231444 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:49.231527 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":222,"threads_started":1}
I20260809 13:13:49.685762 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:49.685014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:60519)
0809 13:13:49.685465 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:49.685554 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":211,"threads_started":1}
I20260809 13:13:49.783279 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:49.782560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:35605)
0809 13:13:49.783023 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:49.783101 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":235,"threads_started":1}
I20260809 13:13:50.233282 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:50.232563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:55325)
0809 13:13:50.233029 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:50.233111 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":228,"threads_started":1}
I20260809 13:13:50.687435 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:50.686692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:44913)
0809 13:13:50.687187 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:50.687267 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":249,"threads_started":1}
I20260809 13:13:50.784849 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:50.784104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:41297)
0809 13:13:50.784577 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:50.784676 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":245,"threads_started":1}
I20260809 13:13:51.234843 5537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:51.234126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:46685)
0809 13:13:51.234579 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:51.234681 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260809 13:13:51.688948 5538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:51.688250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:60675)
0809 13:13:51.688695 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:51.688784 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260809 13:13:51.786518 5540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:51.785734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:40137)
0809 13:13:51.786216 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:51.786297 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":209,"threads_started":1}
I20260809 13:13:52.236497 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:52.235660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:36549)
0809 13:13:52.236171 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:52.236256 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":209,"threads_started":1}
I20260809 13:13:52.690693 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:52.689831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:43681)
0809 13:13:52.690386 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:52.690468 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":268,"threads_started":1}
I20260809 13:13:52.788220 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:52.787381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:45433)
0809 13:13:52.787933 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:52.788039 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":290,"threads_started":1}
I20260809 13:13:53.238168 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:53.237420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:44813)
0809 13:13:53.237900 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:53.237985 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":232,"threads_started":1}
I20260809 13:13:53.692242 5548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:53.691564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:60789)
0809 13:13:53.691986 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:53.692066 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":210,"threads_started":1}
I20260809 13:13:53.789816 5549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:53.789013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:43273)
0809 13:13:53.789520 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:53.789621 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":228,"threads_started":1}
I20260809 13:13:54.239825 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:54.239078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:39775)
0809 13:13:54.239561 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:54.239641 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":220,"threads_started":1}
I20260809 13:13:54.694000 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:54.693149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:49143)
0809 13:13:54.693734 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:54.693822 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":305,"threads_started":1}
I20260809 13:13:54.791424 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:54.790728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:52653)
0809 13:13:54.791176 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:54.791256 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":253,"threads_started":1}
I20260809 13:13:55.241366 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:55.240675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:40555)
0809 13:13:55.241125 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:55.241206 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":249,"threads_started":1}
I20260809 13:13:55.695627 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:55.694930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:41881)
0809 13:13:55.695365 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:55.695446 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":233,"threads_started":1}
I20260809 13:13:55.792980 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:55.792300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:47297)
0809 13:13:55.792718 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:55.792805 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":207,"threads_started":1}
I20260809 13:13:56.242859 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:56.242179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:34789)
0809 13:13:56.242583 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:56.242679 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":197,"threads_started":1}
I20260809 13:13:56.697216 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:56.696502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:43909)
0809 13:13:56.696924 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:56.697013 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":214,"threads_started":1}
I20260809 13:13:56.794548 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:56.793882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:51415)
0809 13:13:56.794315 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:56.794395 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260809 13:13:57.244400 5567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:57.243717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:50061)
0809 13:13:57.244157 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:57.244241 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260809 13:13:57.698834 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:57.698114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:50203)
0809 13:13:57.698586 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:57.698670 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260809 13:13:57.796125 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:57.795379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:51059)
0809 13:13:57.795833 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:57.795925 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":216,"threads_started":1}
I20260809 13:13:58.246125 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:58.245318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:50459)
0809 13:13:58.245831 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:58.245932 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":256,"threads_started":1}
I20260809 13:13:58.700564 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:58.699756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:42207)
0809 13:13:58.700285 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:58.700381 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":274,"threads_started":1}
I20260809 13:13:58.797813 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:58.797014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:51635)
0809 13:13:58.797533 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:58.797628 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":276,"threads_started":1}
I20260809 13:13:59.247737 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:59.246970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:59479)
0809 13:13:59.247453 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:59.247551 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":262,"threads_started":1}
I20260809 13:13:59.702230 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:59.701500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:44757)
0809 13:13:59.701942 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:59.702027 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":220,"threads_started":1}
I20260809 13:13:59.799678 5581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:13:59.798887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.129:35981)
0809 13:13:59.799365 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 13:13:59.799486 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":249,"threads_started":1}
I20260809 13:14:00.249619 5583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:00.248836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.130:44539)
0809 13:14:00.249290 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:00.249388 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":226,"threads_started":1}
I20260809 13:14:00.680464 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 4847
I20260809 13:14:00.700816 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:00.681732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.10.181.129:40953)
0809 13:14:00.682326 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:00.682347 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:00.682397 (+ 50us) server_negotiation.cc:407] Connection header received
0809 13:14:00.683901 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:00.683911 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:00.684342 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:00.684620 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:00.686029 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.687267 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.688493 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.689141 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.689271 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:00.695327 (+ 6056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:00.695335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:00.695349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:00.695386 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:00.698066 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.698669 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.698676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.698682 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.698851 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.699151 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.699154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.699156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.699652 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:00.699875 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:00.700211 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 13:14:00.700537 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":223,"threads_started":1}
I20260809 13:14:00.702193 5098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:00.706336 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:00.703349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:46013 (local address 127.10.181.131:46493)
0809 13:14:00.705983 (+ 2634us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:00.706100 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.190:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2500,"thread_start_us":2389,"threads_started":1}
I20260809 13:14:00.784255 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 4847
I20260809 13:14:00.809719 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 5113
I20260809 13:14:00.826212 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:00.810792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.10.181.130:33033)
0809 13:14:00.811307 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:00.811322 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:00.811361 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:14:00.812873 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:00.812884 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:00.813306 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:00.813665 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:00.814909 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.816164 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.817318 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.818019 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.818153 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:00.820547 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:00.820554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:00.820567 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:00.820603 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:00.823471 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.824053 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.824059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.824065 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.824215 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.824491 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.824494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.824495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.825002 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:00.825232 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:00.825633 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 13:14:00.825934 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":201,"threads_started":1}
I20260809 13:14:00.827488 5303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:00.911216 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 5113
I20260809 13:14:00.937402 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 5315
I20260809 13:14:00.954080 5592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:00.938526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.10.181.131:34933)
0809 13:14:00.939179 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:00.939194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:00.939237 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:14:00.940895 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:00.940905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:00.941330 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:00.941669 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:00.942892 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.944098 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.945268 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:00.945973 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:00.946113 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:00.948492 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:00.948500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:00.948513 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:00.948550 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:00.951347 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.951917 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.951923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.951929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.952094 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:00.952362 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:00.952365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:00.952366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:00.952859 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:00.953079 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:00.953372 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 13:14:00.953784 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":318,"threads_started":1}
I20260809 13:14:00.955353 5503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:01.036381 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 5315
2026-08-09T13:14:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64477 ms)
[----------] 30 tests from SecurityITest (294178 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:01 dist-test-slave-42z9 krb5kdc[5597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:01 dist-test-slave-42z9 krb5kdc[5597](info): set up 2 sockets
Aug 09 13:14:01 dist-test-slave-42z9 krb5kdc[5597](info): commencing operation
krb5kdc: starting...
W20260809 13:14:02.267485 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:02 dist-test-slave-42z9 krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:02.542529 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34473
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:02.867125 5614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:02.867378 5614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:02.867471 5614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:02.874349 5614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:02.874446 5614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:02.874498 5614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:02.874564 5614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:02.874634 5614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:02.884465 5614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:02.885694 5614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:02.886945 5614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:02.898000 5626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:02.898082 5623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:02.898043 5622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:02.899281 5614 server_base.cc:1034] running on GCE node
I20260809 13:14:02.900029 5614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:02.901274 5614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:02.902469 5614 hybrid_clock.cc:648] HybridClock initialized: now 1786281242902420 us; error 35 us; skew 500 ppm
I20260809 13:14:02.902794 5614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:02 dist-test-slave-42z9 krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:02.905712 5614 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:02.907069 5614 webserver.cc:460] Webserver started at http://127.10.181.190:45757/ using document root <none> and password file <none>
I20260809 13:14:02.907711 5614 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:02.907824 5614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:02.908098 5614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:02.910951 5614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "adc56d509f10489c897e6ffcbafbd816"
format_stamp: "Formatted at 2026-08-09 13:14:02 on dist-test-slave-42z9"
server_key: "3fc234d4ee717b242e7ca78d6bc6e065"
server_key_iv: "987bf81af73e2644151fd4b9c852e6f1"
server_key_version: "encryptionkey@0"
I20260809 13:14:02.911648 5614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "adc56d509f10489c897e6ffcbafbd816"
format_stamp: "Formatted at 2026-08-09 13:14:02 on dist-test-slave-42z9"
server_key: "3fc234d4ee717b242e7ca78d6bc6e065"
server_key_iv: "987bf81af73e2644151fd4b9c852e6f1"
server_key_version: "encryptionkey@0"
I20260809 13:14:02.917292 5614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:14:02.920544 5635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:02.921795 5614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:02.921952 5614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "adc56d509f10489c897e6ffcbafbd816"
format_stamp: "Formatted at 2026-08-09 13:14:02 on dist-test-slave-42z9"
server_key: "3fc234d4ee717b242e7ca78d6bc6e065"
server_key_iv: "987bf81af73e2644151fd4b9c852e6f1"
server_key_version: "encryptionkey@0"
I20260809 13:14:02.922118 5614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:02.949779 5614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:02.953822 5614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:02.954092 5614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:02.971189 5614 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34473
I20260809 13:14:02.971217 5719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34473 every 8 connection(s)
I20260809 13:14:02.972493 5614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:02.974785 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 5614
I20260809 13:14:02.975014 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:02.975859 10966 external_mini_cluster.cc:1428] Setting key 15e81efec45b510e04568da741ecca4f
I20260809 13:14:02.976758 5720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:14:02 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:02.991043 5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: Bootstrap starting.
I20260809 13:14:02.994369 5720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:02.995414 5720 log.cc:824] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:02.998682 5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: No bootstrap required, opened a new log
I20260809 13:14:02.999457 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:02.977966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.10.181.190:34473)
0809 13:14:02.978726 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:02.978737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:02.978778 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:02.980295 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:02.980315 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:02.980665 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:02.980994 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:02.982806 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:02.984188 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:02.986192 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:02.986875 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:02.987012 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:02.992159 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:02.992166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:02.992180 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:02.992219 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:02.995300 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:02.996281 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:02.996290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:02.996298 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:02.996525 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:02.996903 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:02.996909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:02.996912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:02.997402 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:02.997683 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:02.998121 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 13:14:02.998414 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":337,"threads_started":1}
I20260809 13:14:03.005961 5720 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } }
I20260809 13:14:03.006297 5720 raft_consensus.cc:374] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:03.006393 5720 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc56d509f10489c897e6ffcbafbd816, State: Initialized, Role: FOLLOWER
I20260809 13:14:03.006914 5720 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } }
I20260809 13:14:03.007125 5720 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:03.007253 5720 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:03.007401 5720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:03.009410 5720 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } }
I20260809 13:14:03.009918 5720 leader_election.cc:302] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc56d509f10489c897e6ffcbafbd816; no voters:
I20260809 13:14:03.010448 5720 leader_election.cc:288] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:03.010561 5728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:03.011220 5728 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [term 1 LEADER]: Becoming Leader. State: Replica: adc56d509f10489c897e6ffcbafbd816, State: Running, Role: LEADER
I20260809 13:14:03.011763 5728 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } }
I20260809 13:14:03.012403 5720 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:03.014675 5734 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc56d509f10489c897e6ffcbafbd816. Latest consensus state: current_term: 1 leader_uuid: "adc56d509f10489c897e6ffcbafbd816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } } }
I20260809 13:14:03.014703 5731 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adc56d509f10489c897e6ffcbafbd816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc56d509f10489c897e6ffcbafbd816" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34473 } } }
I20260809 13:14:03.015272 5734 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:03.015280 5731 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:03.016598 5738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:03.021284 5738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:03.029148 5738 catalog_manager.cc:1347] Generated new cluster ID: bbe14c791ea24fa9a4c428644906ee53
I20260809 13:14:03.029245 5738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:03.054806 5738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:03.056559 5738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:03.066304 5738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: Generated new TSK 0
I20260809 13:14:03.067207 5738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:14:03.134517 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:03.522473 5765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:03.522748 5765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:03.522883 5765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:03.530155 5765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:03.530267 5765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:03.530416 5765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:03.539692 5765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:03.540843 5765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:03.542111 5765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:03.554812 5774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:03.554852 5778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:03.554883 5771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:03.558665 5765 server_base.cc:1034] running on GCE node
I20260809 13:14:03.559222 5765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:03.560127 5765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:03.561323 5765 hybrid_clock.cc:648] HybridClock initialized: now 1786281243561274 us; error 41 us; skew 500 ppm
I20260809 13:14:03.561654 5765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:03 dist-test-slave-42z9 krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:03.564581 5765 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:14:03.566087 5765 webserver.cc:460] Webserver started at http://127.10.181.129:32999/ using document root <none> and password file <none>
I20260809 13:14:03.566771 5765 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:03.566887 5765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:03.567184 5765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:03.570041 5765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "fcb2059ca2c44ff19579c077fdb1f9dc"
format_stamp: "Formatted at 2026-08-09 13:14:03 on dist-test-slave-42z9"
server_key: "dab31a8780594c9ffc55eee75b062a21"
server_key_iv: "2d76c394cef9c5ed2a1ae98fdf7ac302"
server_key_version: "encryptionkey@0"
I20260809 13:14:03.570780 5765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "fcb2059ca2c44ff19579c077fdb1f9dc"
format_stamp: "Formatted at 2026-08-09 13:14:03 on dist-test-slave-42z9"
server_key: "dab31a8780594c9ffc55eee75b062a21"
server_key_iv: "2d76c394cef9c5ed2a1ae98fdf7ac302"
server_key_version: "encryptionkey@0"
I20260809 13:14:03.575613 5765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:14:03.578925 5787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:03.580147 5765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:14:03.580291 5765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "fcb2059ca2c44ff19579c077fdb1f9dc"
format_stamp: "Formatted at 2026-08-09 13:14:03 on dist-test-slave-42z9"
server_key: "dab31a8780594c9ffc55eee75b062a21"
server_key_iv: "2d76c394cef9c5ed2a1ae98fdf7ac302"
server_key_version: "encryptionkey@0"
I20260809 13:14:03.580474 5765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:03.600005 5765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:03.603511 5765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:03.603773 5765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:03.604642 5765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:03.605934 5765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:03.606014 5765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:03.606137 5765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:03.606189 5765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:03.636746 5765 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:37327
I20260809 13:14:03.636806 5951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:37327 every 8 connection(s)
I20260809 13:14:03.637984 5765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:03.643960 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 5765
I20260809 13:14:03.644181 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:03.645008 10966 external_mini_cluster.cc:1428] Setting key f09930adaa7366b5d67fc4cd712c000b
Aug 09 13:14:03 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:03.658071 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:03.640378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:37011 (local address 127.10.181.190:34473)
0809 13:14:03.640872 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:03.640878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:03.641780 (+ 902us) server_negotiation.cc:407] Connection header received
0809 13:14:03.643510 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:03.643514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:03.643582 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:03.643683 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:03.645668 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:03.646499 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:03.648537 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:03.648932 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:03.649031 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:03.652657 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:03.652662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:03.652665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:03.652699 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:03.655144 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:03.655829 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:03.655833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:03.655835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:03.655957 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:03.656379 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:03.656383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:03.656385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:03.656649 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:03.656801 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:03.657490 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 13:14:03.657726 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":235,"threads_started":1}
I20260809 13:14:03.658890 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:03.640912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34473 (local address 127.10.181.129:37011)
0809 13:14:03.641636 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:03.641674 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:03.643272 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:03.643955 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:03.643966 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:03.644685 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:03.645482 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:03.645512 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:03.646663 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:03.646669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:03.648398 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:03.648415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:03.649161 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:03.649170 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:03.649463 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:03.650119 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:03.650151 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:03.652447 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:03.655277 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:03.655286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:03.655300 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:03.655706 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:03.656062 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:03.656067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:03.656071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:03.656262 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:03.656778 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:03.656785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:03.657078 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 13:14:03.657312 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 13:14:03.657806 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":262,"threads_started":1}
I20260809 13:14:03.660287 5955 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34473
I20260809 13:14:03.660585 5955 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:03.661296 5955 heartbeater.cc:511] Master 127.10.181.190:34473 requested a full tablet report, sending...
I20260809 13:14:03.663523 5666 ts_manager.cc:194] Registered new tserver with Master: fcb2059ca2c44ff19579c077fdb1f9dc (127.10.181.129:37327)
I20260809 13:14:03.665375 5666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:37011
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:14:03.709800 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:04.038836 5961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:04.039098 5961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:04.039197 5961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:04.046257 5961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:04.046355 5961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:04.046478 5961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:04.055771 5961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:04.056916 5961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:04.058159 5961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:04.070374 5974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:04.070423 5967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:04.070484 5971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:04.071421 5961 server_base.cc:1034] running on GCE node
I20260809 13:14:04.072003 5961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:04.072860 5961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:04.074051 5961 hybrid_clock.cc:648] HybridClock initialized: now 1786281244073996 us; error 45 us; skew 500 ppm
I20260809 13:14:04.074339 5961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:04 dist-test-slave-42z9 krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:04.077365 5961 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:14:04.078794 5961 webserver.cc:460] Webserver started at http://127.10.181.130:42723/ using document root <none> and password file <none>
I20260809 13:14:04.079437 5961 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:04.079559 5961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:04.079859 5961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:04.082741 5961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "2b9b14d8b2a2485ebc67707bbc507e22"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "6d5cc25c85eea15966e35da0690515ab"
server_key_iv: "27f126be28fc7583b04bf513314e91e1"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.083525 5961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "2b9b14d8b2a2485ebc67707bbc507e22"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "6d5cc25c85eea15966e35da0690515ab"
server_key_iv: "27f126be28fc7583b04bf513314e91e1"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.088464 5961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:14:04.091890 5984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.093151 5961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:14:04.093297 5961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "2b9b14d8b2a2485ebc67707bbc507e22"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "6d5cc25c85eea15966e35da0690515ab"
server_key_iv: "27f126be28fc7583b04bf513314e91e1"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.093515 5961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:04.132817 5961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:04.136638 5961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:04.136906 5961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:04.137902 5961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:04.139225 5961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:04.139303 5961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.139441 5961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:04.139501 5961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.170051 5961 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:41743
I20260809 13:14:04.170097 6170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:41743 every 8 connection(s)
I20260809 13:14:04.171352 5961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:04.176486 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 5961
I20260809 13:14:04.176659 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:04.177119 10966 external_mini_cluster.cc:1428] Setting key 4776e876afc48b734cc9778a432f3f81
Aug 09 13:14:04 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:04.190977 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.173897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:49665 (local address 127.10.181.190:34473)
0809 13:14:04.174353 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.174357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.175297 (+ 940us) server_negotiation.cc:407] Connection header received
0809 13:14:04.177179 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.177186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.177275 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.177397 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:04.178885 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.179617 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.181141 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.181628 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.181741 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.185307 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:04.185312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:04.185316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:04.185350 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:04.187878 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.188614 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.188625 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.188628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.188763 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.189193 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.189197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.189199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.189508 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:04.189655 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.190369 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.190596 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":232,"threads_started":1}
I20260809 13:14:04.192060 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.174444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34473 (local address 127.10.181.130:49665)
0809 13:14:04.175136 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.175183 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.176909 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.177630 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.177638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.178142 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:04.178710 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.178728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.179769 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.179773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.180984 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.180997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.181861 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.181868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.182082 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.182711 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:04.182734 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:04.185050 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:04.188016 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:04.188026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:04.188044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:04.188497 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:04.188873 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:04.188878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:04.188882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:04.189073 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:04.189641 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:04.189648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:04.189993 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.190251 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.190774 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":265,"threads_started":1}
I20260809 13:14:04.193847 6171 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34473
I20260809 13:14:04.194164 6171 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:04.194883 6171 heartbeater.cc:511] Master 127.10.181.190:34473 requested a full tablet report, sending...
I20260809 13:14:04.196396 5666 ts_manager.cc:194] Registered new tserver with Master: 2b9b14d8b2a2485ebc67707bbc507e22 (127.10.181.130:41743)
I20260809 13:14:04.197232 5666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:49665
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:14:04.242281 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:04.573729 6179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:04.574005 6179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:04.574148 6179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:04.581207 6179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:04.581316 6179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:04.581487 6179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:04.590953 6179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:04.592171 6179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:04.593495 6179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:04.605997 6184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:04.606025 6185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:04.606150 6188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:04.606976 6179 server_base.cc:1034] running on GCE node
I20260809 13:14:04.607555 6179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:04.608374 6179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:04.609543 6179 hybrid_clock.cc:648] HybridClock initialized: now 1786281244609488 us; error 36 us; skew 500 ppm
I20260809 13:14:04.609827 6179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:04 dist-test-slave-42z9 krb5kdc[5597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:04.612792 6179 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:14:04.614240 6179 webserver.cc:460] Webserver started at http://127.10.181.131:42807/ using document root <none> and password file <none>
I20260809 13:14:04.614864 6179 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:04.614981 6179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:04.615274 6179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:04.618063 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "02dd356bfcf042708d5b6d15f28c40f1"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "d86447288539e7cf58379d28082f6f03"
server_key_iv: "51a5376eae91c7c0a05873c00f041feb"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.618794 6179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "02dd356bfcf042708d5b6d15f28c40f1"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "d86447288539e7cf58379d28082f6f03"
server_key_iv: "51a5376eae91c7c0a05873c00f041feb"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.623721 6179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:14:04.627243 6197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.628464 6179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:14:04.628605 6179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "02dd356bfcf042708d5b6d15f28c40f1"
format_stamp: "Formatted at 2026-08-09 13:14:04 on dist-test-slave-42z9"
server_key: "d86447288539e7cf58379d28082f6f03"
server_key_iv: "51a5376eae91c7c0a05873c00f041feb"
server_key_version: "encryptionkey@0"
I20260809 13:14:04.628801 6179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:04.648689 6179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:04.652738 6179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:04.653002 6179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:04.654026 6179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:04.655359 6179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:04.655437 6179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.655555 6179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:04.655606 6179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:04.668232 5955 heartbeater.cc:503] Master 127.10.181.190:34473 was elected leader, sending a full tablet report...
I20260809 13:14:04.686445 6179 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:41939
I20260809 13:14:04.686496 6368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:41939 every 8 connection(s)
I20260809 13:14:04.687729 6179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:04.690917 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 6179
I20260809 13:14:04.691094 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:04.691520 10966 external_mini_cluster.cc:1428] Setting key f24e6d02af13cde5721db70222054529
Aug 09 13:14:04 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:04.707067 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.690218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:34093 (local address 127.10.181.190:34473)
0809 13:14:04.690401 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.690405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.691709 (+ 1304us) server_negotiation.cc:407] Connection header received
0809 13:14:04.693693 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.693698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.693774 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.693875 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:04.695295 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.696107 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.697638 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.698055 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.698168 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.701472 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:04.701479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:04.701483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:04.701516 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:04.703786 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.704541 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.704544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.704546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.704658 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.705173 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.705177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.705180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.705526 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:04.705690 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.706536 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.706758 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:14:04.707922 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.690729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34473 (local address 127.10.181.131:34093)
0809 13:14:04.691523 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.691579 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.693416 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.694068 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.694077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.694575 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:04.695134 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.695152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.696227 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.696231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.697517 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.697530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.698250 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.698256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.698451 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.699066 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:04.699094 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:04.701260 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:04.703905 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:04.703913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:04.703924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:04.704319 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:04.704803 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:04.704808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:04.704811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:04.705062 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:04.705677 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:04.705684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:04.706089 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.706410 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.706847 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":305,"threads_started":1}
I20260809 13:14:04.709120 6369 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34473
I20260809 13:14:04.709384 6369 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:04.710080 6369 heartbeater.cc:511] Master 127.10.181.190:34473 requested a full tablet report, sending...
I20260809 13:14:04.711411 5666 ts_manager.cc:194] Registered new tserver with Master: 02dd356bfcf042708d5b6d15f28c40f1 (127.10.181.131:41939)
I20260809 13:14:04.712167 5666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:34093
I20260809 13:14:04.718885 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:04.735949 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.723146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.10.181.190:34473)
0809 13:14:04.723331 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.723338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.723734 (+ 396us) server_negotiation.cc:407] Connection header received
0809 13:14:04.723868 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.723872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.723938 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.724006 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:04.725397 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.726173 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.727436 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.727809 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.727904 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.730386 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:04.730391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:04.730395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:04.730426 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:04.733069 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.734013 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.734018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.734020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.734145 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:04.734503 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:04.734507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:04.734509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:04.734834 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:04.735075 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.735416 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.735628 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:14:04.739739 5666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:04.746505 5666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:04.769178 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.758136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41743 (local address 127.0.0.1:48010)
0809 13:14:04.759177 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.759215 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.759399 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.760717 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.760726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.760760 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:04.761178 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.761191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.763288 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.763295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.765805 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.765829 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.768138 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.768145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.768372 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.768426 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.768566 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.768767 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":623,"threads_started":1}
I20260809 13:14:04.770181 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.758327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.10.181.130:41743)
0809 13:14:04.759807 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.759816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.759844 (+ 28us) server_negotiation.cc:407] Connection header received
0809 13:14:04.759946 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.759955 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.760216 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.760416 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:04.761413 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.763105 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.766035 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.767603 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.767793 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.767978 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.769614 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.769828 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310,"thread_start_us":1152,"threads_started":1}
I20260809 13:14:04.774976 6056 tablet_service.cc:1467] Processing CreateTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (DEFAULT_TABLE table=test-table [id=c8f50870208c484893c4441aed3bdb65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:04.776675 6056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c7b1cb1a4b4f2d88da1be7d7384563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:04.779757 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.759221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:37327 (local address 127.0.0.1:45640)
0809 13:14:04.770901 (+ 11680us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.770920 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.771018 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.773900 (+ 2882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.773905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.773930 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:04.774074 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.774084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.775785 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.775790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.776947 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.776959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.778902 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.778908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.779127 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.779220 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.779308 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.779459 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11580,"thread_start_us":6770,"threads_started":1}
I20260809 13:14:04.779873 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.760095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.10.181.129:37327)
0809 13:14:04.773009 (+ 12914us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.773019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.773050 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:14:04.773160 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.773169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.773403 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.773637 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:04.774222 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.775615 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.777094 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.778458 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.778649 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.778808 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.779391 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.779566 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12725,"thread_start_us":13402,"threads_started":1}
I20260809 13:14:04.782124 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.758421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41939 (local address 127.0.0.1:56500)
0809 13:14:04.766254 (+ 7833us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.766274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.766393 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.774786 (+ 8393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.774790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.774816 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:04.775243 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.775258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.777384 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.777389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.779717 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.779731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.781371 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.781377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.781569 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.781602 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.781676 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.781795 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7711,"thread_start_us":13082,"threads_started":1}
I20260809 13:14:04.782182 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.759649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.10.181.131:41939)
0809 13:14:04.774186 (+ 14537us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.774194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.774234 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:14:04.774329 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.774336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.774574 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.775443 (+ 869us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:04.775731 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.777246 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.779908 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.781190 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.781366 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.781559 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.781739 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.781892 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14360,"thread_start_us":14220,"threads_started":1}
I20260809 13:14:04.783818 5858 tablet_service.cc:1467] Processing CreateTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (DEFAULT_TABLE table=test-table [id=c8f50870208c484893c4441aed3bdb65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:04.785302 5858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c7b1cb1a4b4f2d88da1be7d7384563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:04.785763 6272 tablet_service.cc:1467] Processing CreateTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (DEFAULT_TABLE table=test-table [id=c8f50870208c484893c4441aed3bdb65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:04.786888 6272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c7b1cb1a4b4f2d88da1be7d7384563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:04.790949 6392 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Bootstrap starting.
I20260809 13:14:04.794695 6392 tablet_bootstrap.cc:654] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:04.795981 6392 log.cc:824] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:04.798501 6397 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Bootstrap starting.
I20260809 13:14:04.799726 6392 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: No bootstrap required, opened a new log
I20260809 13:14:04.800081 6392 ts_tablet_manager.cc:1397] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.006s
I20260809 13:14:04.800144 6399 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Bootstrap starting.
I20260809 13:14:04.802995 6397 tablet_bootstrap.cc:654] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:04.803648 6399 tablet_bootstrap.cc:654] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:04.804267 6397 log.cc:824] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:04.804863 6399 log.cc:824] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:04.806872 6397 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: No bootstrap required, opened a new log
I20260809 13:14:04.807132 6397 ts_tablet_manager.cc:1397] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 13:14:04.808053 6392 raft_consensus.cc:348] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.808502 6392 raft_consensus.cc:374] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:04.808619 6392 raft_consensus.cc:729] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9b14d8b2a2485ebc67707bbc507e22, State: Initialized, Role: FOLLOWER
I20260809 13:14:04.809365 6399 tablet_bootstrap.cc:492] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: No bootstrap required, opened a new log
I20260809 13:14:04.809252 6392 consensus_queue.cc:260] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.809703 6399 ts_tablet_manager.cc:1397] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 13:14:04.812799 6171 heartbeater.cc:503] Master 127.10.181.190:34473 was elected leader, sending a full tablet report...
I20260809 13:14:04.813418 6392 ts_tablet_manager.cc:1428] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 13:14:04.816496 6397 raft_consensus.cc:348] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.816931 6397 raft_consensus.cc:374] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:04.817037 6397 raft_consensus.cc:729] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb2059ca2c44ff19579c077fdb1f9dc, State: Initialized, Role: FOLLOWER
I20260809 13:14:04.817684 6397 consensus_queue.cc:260] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.819267 6397 ts_tablet_manager.cc:1428] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260809 13:14:04.819419 6399 raft_consensus.cc:348] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.819888 6399 raft_consensus.cc:374] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:04.820050 6399 raft_consensus.cc:729] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02dd356bfcf042708d5b6d15f28c40f1, State: Initialized, Role: FOLLOWER
I20260809 13:14:04.820894 6399 consensus_queue.cc:260] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.821960 6369 heartbeater.cc:503] Master 127.10.181.190:34473 was elected leader, sending a full tablet report...
I20260809 13:14:04.822556 6399 ts_tablet_manager.cc:1428] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Time spent starting tablet: real 0.013s user 0.009s sys 0.002s
W20260809 13:14:04.890226 5956 tablet.cc:2369] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:04.923295 6172 tablet.cc:2369] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:04.939266 6372 tablet.cc:2369] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:04.950724 6403 raft_consensus.cc:482] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:04.951011 6403 raft_consensus.cc:504] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.952910 6403 leader_election.cc:288] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02dd356bfcf042708d5b6d15f28c40f1 (127.10.181.131:41939), fcb2059ca2c44ff19579c077fdb1f9dc (127.10.181.129:37327)
I20260809 13:14:04.959686 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.953351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:39137 (local address 127.10.181.131:41939)
0809 13:14:04.953600 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.953606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.954283 (+ 677us) server_negotiation.cc:407] Connection header received
0809 13:14:04.954511 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.954516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.954578 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.954651 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:04.955262 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.956001 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.957830 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.958704 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.958864 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.958936 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.959278 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.959415 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:14:04.959734 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.953136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:41939 (local address 127.10.181.130:39137)
0809 13:14:04.954168 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.954187 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.954345 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.954691 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.954695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.954719 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:04.955109 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.955121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.956133 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.956137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.957677 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.957689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.958922 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.958929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.959100 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.959144 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.959233 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.959363 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":621,"threads_started":1}
I20260809 13:14:04.960117 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.953594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:38967 (local address 127.10.181.129:37327)
0809 13:14:04.953802 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:04.953807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:04.954363 (+ 556us) server_negotiation.cc:407] Connection header received
0809 13:14:04.954544 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:04.954547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:04.954607 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:04.954689 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:04.955311 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.956046 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.957877 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.958898 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.959050 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.959121 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:04.959752 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 13:14:04.959874 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:14:04.960196 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:04.953207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:37327 (local address 127.10.181.130:38967)
0809 13:14:04.954273 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:04.954288 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:04.954399 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:04.954720 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:04.954725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:04.954749 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:04.955161 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.955169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.956187 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.956191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.957724 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:04.957737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:04.959413 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:04.959421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:04.959593 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:04.959617 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 13:14:04.959704 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 13:14:04.959823 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":41,"thread_start_us":761,"threads_started":1}
I20260809 13:14:04.961014 6305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c7b1cb1a4b4f2d88da1be7d7384563" candidate_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02dd356bfcf042708d5b6d15f28c40f1" is_pre_election: true
I20260809 13:14:04.961130 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c7b1cb1a4b4f2d88da1be7d7384563" candidate_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" is_pre_election: true
I20260809 13:14:04.961643 6305 raft_consensus.cc:2454] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9b14d8b2a2485ebc67707bbc507e22 in term 0.
I20260809 13:14:04.961702 5891 raft_consensus.cc:2454] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b9b14d8b2a2485ebc67707bbc507e22 in term 0.
I20260809 13:14:04.962370 5986 leader_election.cc:302] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9b14d8b2a2485ebc67707bbc507e22, fcb2059ca2c44ff19579c077fdb1f9dc; no voters:
I20260809 13:14:04.962757 6403 raft_consensus.cc:2781] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:04.962882 6403 raft_consensus.cc:482] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:04.962985 6403 raft_consensus.cc:3037] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:04.965147 6403 raft_consensus.cc:504] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.965821 6403 leader_election.cc:288] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [CANDIDATE]: Term 1 election: Requested vote from peers 02dd356bfcf042708d5b6d15f28c40f1 (127.10.181.131:41939), fcb2059ca2c44ff19579c077fdb1f9dc (127.10.181.129:37327)
I20260809 13:14:04.966174 6305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c7b1cb1a4b4f2d88da1be7d7384563" candidate_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02dd356bfcf042708d5b6d15f28c40f1"
I20260809 13:14:04.966374 6305 raft_consensus.cc:3037] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:04.966475 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c7b1cb1a4b4f2d88da1be7d7384563" candidate_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc"
I20260809 13:14:04.966660 5891 raft_consensus.cc:3037] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:04.968502 6305 raft_consensus.cc:2454] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9b14d8b2a2485ebc67707bbc507e22 in term 1.
I20260809 13:14:04.968871 5988 leader_election.cc:302] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02dd356bfcf042708d5b6d15f28c40f1, 2b9b14d8b2a2485ebc67707bbc507e22; no voters:
I20260809 13:14:04.968969 5891 raft_consensus.cc:2454] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b9b14d8b2a2485ebc67707bbc507e22 in term 1.
I20260809 13:14:04.969125 6403 raft_consensus.cc:2781] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:04.969609 6403 raft_consensus.cc:686] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9b14d8b2a2485ebc67707bbc507e22, State: Running, Role: LEADER
I20260809 13:14:04.970104 6403 consensus_queue.cc:237] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } }
I20260809 13:14:04.974709 5666 catalog_manager.cc:5495] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9b14d8b2a2485ebc67707bbc507e22 (127.10.181.130). New cstate: current_term: 1 leader_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9b14d8b2a2485ebc67707bbc507e22" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 } health_report { overall_health: UNKNOWN } } }
I20260809 13:14:05.016379 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:05.011409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.10.181.130:41743)
0809 13:14:05.011624 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:05.011630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:05.011648 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:14:05.011800 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:05.011804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:05.011882 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:05.012001 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:05.012347 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.013098 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.014452 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.014871 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.014966 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:05.015374 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:05.015848 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:05.015952 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:05.016045 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 13:14:05.016109 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 13:14:05.024600 6305 raft_consensus.cc:1261] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 1 FOLLOWER]: Refusing update from remote peer 2b9b14d8b2a2485ebc67707bbc507e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:05.024601 5891 raft_consensus.cc:1261] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 1 FOLLOWER]: Refusing update from remote peer 2b9b14d8b2a2485ebc67707bbc507e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:05.025707 6411 consensus_queue.cc:1035] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02dd356bfcf042708d5b6d15f28c40f1" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 41939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:05.026054 6403 consensus_queue.cc:1035] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb2059ca2c44ff19579c077fdb1f9dc" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:05.039088 6423 mvcc.cc:205] Tried to move back new op lower bound from 7316607979609903104 to 7316607979401007104. Current Snapshot: MvccSnapshot[applied={T|T < 7316607979609903104}]
I20260809 13:14:05.053072 5666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:05.053364 5666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:05.057026 5666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adc56d509f10489c897e6ffcbafbd816: Sending DeleteTablet for 3 replicas of tablet 92c7b1cb1a4b4f2d88da1be7d7384563
I20260809 13:14:05.058163 6056 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:05 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:48010
I20260809 13:14:05.058189 6272 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:05 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:56500
I20260809 13:14:05.058459 5858 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c7b1cb1a4b4f2d88da1be7d7384563 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:05 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:45640
I20260809 13:14:05.060457 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 5765
I20260809 13:14:05.069657 6435 tablet_replica.cc:330] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: stopping tablet replica
I20260809 13:14:05.070142 6435 raft_consensus.cc:2229] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:05.070679 6435 raft_consensus.cc:2258] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:05.071486 6436 tablet_replica.cc:330] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: stopping tablet replica
I20260809 13:14:05.071882 6436 raft_consensus.cc:2229] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:05.072352 6436 raft_consensus.cc:2258] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:05.074788 6435 ts_tablet_manager.cc:1905] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:05.081570 6435 ts_tablet_manager.cc:1918] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:05.081804 6435 log.cc:1196] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/92c7b1cb1a4b4f2d88da1be7d7384563
I20260809 13:14:05.082290 6436 ts_tablet_manager.cc:1905] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:05.082415 6435 ts_tablet_manager.cc:1939] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 02dd356bfcf042708d5b6d15f28c40f1: Deleting consensus metadata
I20260809 13:14:05.084368 5645 catalog_manager.cc:4841] TS 02dd356bfcf042708d5b6d15f28c40f1 (127.10.181.131:41939): tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (table test-table [id=c8f50870208c484893c4441aed3bdb65]) successfully deleted
I20260809 13:14:05.085461 6434 tablet_replica.cc:330] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: stopping tablet replica
I20260809 13:14:05.085889 6434 raft_consensus.cc:2229] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:05.086576 6434 raft_consensus.cc:2258] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:05.088778 6434 ts_tablet_manager.cc:1905] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 13:14:05 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:14:05.090840 6436 ts_tablet_manager.cc:1918] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:05.091075 6436 log.cc:1196] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/92c7b1cb1a4b4f2d88da1be7d7384563
I20260809 13:14:05.091730 6436 ts_tablet_manager.cc:1939] T 92c7b1cb1a4b4f2d88da1be7d7384563 P fcb2059ca2c44ff19579c077fdb1f9dc: Deleting consensus metadata
I20260809 13:14:05.093648 5642 catalog_manager.cc:4841] TS fcb2059ca2c44ff19579c077fdb1f9dc (127.10.181.129:37327): tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (table test-table [id=c8f50870208c484893c4441aed3bdb65]) successfully deleted
I20260809 13:14:05.096956 6434 ts_tablet_manager.cc:1918] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:05.097203 6434 log.cc:1196] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/92c7b1cb1a4b4f2d88da1be7d7384563
I20260809 13:14:05.097589 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:05.061845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.10.181.129:37327)
0809 13:14:05.062025 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:05.062031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:05.075333 (+ 13302us) server_negotiation.cc:407] Connection header received
0809 13:14:05.075779 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:05.075785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:05.075869 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:05.075935 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:05.077670 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.078673 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.082698 (+ 4025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.083284 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.083403 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:05.091343 (+ 7940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:05.091360 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:05.091379 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:05.091432 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:05.094793 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.095449 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.095457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.095459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.095590 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.095961 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.095965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.095967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.096303 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:05.096592 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:05.096985 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 13:14:05.097202 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:14:05.097817 6434 ts_tablet_manager.cc:1939] T 92c7b1cb1a4b4f2d88da1be7d7384563 P 2b9b14d8b2a2485ebc67707bbc507e22: Deleting consensus metadata
I20260809 13:14:05.098778 5946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:05.099681 5641 catalog_manager.cc:4841] TS 2b9b14d8b2a2485ebc67707bbc507e22 (127.10.181.130:41743): tablet 92c7b1cb1a4b4f2d88da1be7d7384563 (table test-table [id=c8f50870208c484893c4441aed3bdb65]) successfully deleted
I20260809 13:14:05.192247 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 5765
I20260809 13:14:05.216884 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 5961
Aug 09 13:14:05 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:14:05.238029 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:05.217808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.10.181.130:41743)
0809 13:14:05.218009 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:05.218015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:05.218040 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:14:05.218108 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:05.218113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:05.218192 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:05.218315 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:05.219580 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.220859 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.222183 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.222805 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.222934 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:05.231587 (+ 8653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:05.231602 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:05.231620 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:05.231663 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:05.234951 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.235796 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.235804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.235807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.235967 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.236404 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.236409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.236412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.236820 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:05.237081 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:05.237428 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:14:05.237685 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:14:05.238744 6162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:05.355175 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 5961
I20260809 13:14:05.383234 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 6179
Aug 09 13:14:05 dist-test-slave-42z9 krb5kdc[5597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:14:05.397192 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:05.383966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.10.181.131:41939)
0809 13:14:05.384133 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:05.384137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:05.384185 (+ 48us) server_negotiation.cc:407] Connection header received
0809 13:14:05.384362 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:05.384367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:05.384433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:05.384519 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:05.385810 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.386531 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.387690 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:05.388062 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:05.388147 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:05.392163 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:05.392179 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:05.392197 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:05.392235 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:05.394905 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.395491 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.395498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.395500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.395630 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:05.395906 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:05.395909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:05.395911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:05.396252 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:05.396468 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:05.396704 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 13:14:05.396863 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:14:05.397856 6362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:05.485658 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 6179
I20260809 13:14:05.507093 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 5614
I20260809 13:14:05.508219 5714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:05.615852 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 5614
2026-08-09T13:14:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4583 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T13:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:05Z Disabled control of system clock
I20260809 13:14:05.711314 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34829
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:06.045287 6450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:06.045606 6450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:06.045747 6450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:06.053211 6450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:06.053304 6450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:06.053362 6450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:06.053460 6450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:06.053532 6450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:06.062409 6450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:06.063630 6450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:06.065366 6450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:06.076485 6456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:06.076603 6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:06.076642 6461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:06.077701 6450 server_base.cc:1034] running on GCE node
I20260809 13:14:06.078475 6450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:06.079838 6450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:06.081089 6450 hybrid_clock.cc:648] HybridClock initialized: now 1786281246081041 us; error 37 us; skew 500 ppm
I20260809 13:14:06.081482 6450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:06.087229 6450 webserver.cc:460] Webserver started at http://127.10.181.190:39679/ using document root <none> and password file <none>
I20260809 13:14:06.087916 6450 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:06.088016 6450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:06.088333 6450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:06.091253 6450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "4933fc14629f490ab96fca80922a234d"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "ed3aebcc2815237308ff4ac25972a29e"
server_key_iv: "765855f038dc2783eb523535fad1b483"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.092043 6450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "4933fc14629f490ab96fca80922a234d"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "ed3aebcc2815237308ff4ac25972a29e"
server_key_iv: "765855f038dc2783eb523535fad1b483"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.097088 6450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:14:06.100419 6474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:06.101717 6450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:14:06.101857 6450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "4933fc14629f490ab96fca80922a234d"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "ed3aebcc2815237308ff4ac25972a29e"
server_key_iv: "765855f038dc2783eb523535fad1b483"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.102039 6450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:06.127588 6450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:06.128424 6450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:06.128675 6450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:06.146665 6450 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34829
I20260809 13:14:06.146687 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34829 every 8 connection(s)
I20260809 13:14:06.147959 6450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:06.151650 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:06.157661 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 6450
I20260809 13:14:06.157837 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:06.158406 10966 external_mini_cluster.cc:1428] Setting key c710c1e6023f095922d560e8735888b4
I20260809 13:14:06.162390 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: Bootstrap starting.
I20260809 13:14:06.165596 6557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:06.166549 6557 log.cc:824] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:06.169473 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: No bootstrap required, opened a new log
I20260809 13:14:06.171725 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:06.160332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.10.181.190:34829)
0809 13:14:06.161243 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:06.161254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:06.161296 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:14:06.162982 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:06.163004 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:06.163413 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:06.163734 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:06.164318 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.165620 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.167038 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.167792 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.167990 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:06.169053 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:06.169066 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:06.169082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:06.169788 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:06.169889 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:06.169935 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:06.170057 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:06.170247 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 13:14:06.170555 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":318,"threads_started":1}
I20260809 13:14:06.177302 6557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } }
I20260809 13:14:06.177655 6557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:06.177731 6557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4933fc14629f490ab96fca80922a234d, State: Initialized, Role: FOLLOWER
I20260809 13:14:06.178229 6557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } }
I20260809 13:14:06.178448 6557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:06.178548 6557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:06.178669 6557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:06.180682 6557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } }
I20260809 13:14:06.181146 6557 leader_election.cc:302] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4933fc14629f490ab96fca80922a234d; no voters:
I20260809 13:14:06.181756 6557 leader_election.cc:288] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:06.181872 6572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:06.182507 6572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [term 1 LEADER]: Becoming Leader. State: Replica: 4933fc14629f490ab96fca80922a234d, State: Running, Role: LEADER
I20260809 13:14:06.183032 6572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } }
I20260809 13:14:06.183902 6557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:06.185806 6573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4933fc14629f490ab96fca80922a234d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } } }
I20260809 13:14:06.186213 6573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:06.186187 6574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4933fc14629f490ab96fca80922a234d. Latest consensus state: current_term: 1 leader_uuid: "4933fc14629f490ab96fca80922a234d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4933fc14629f490ab96fca80922a234d" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34829 } } }
I20260809 13:14:06.189283 6574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:06.190464 6581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:06.195637 6581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:06.204177 6581 catalog_manager.cc:1347] Generated new cluster ID: 2beddaf7bb6c411c8665a19139365da8
I20260809 13:14:06.204296 6581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:06.216976 6581 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:06.218227 6581 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:06.230001 6581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: Generated new TSK 0
I20260809 13:14:06.230835 6581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:06.240096 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--builtin_ntp_servers=127.10.181.148:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:06.585923 6594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:06.586196 6594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:06.586323 6594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:06.593421 6594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:06.593523 6594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:06.593662 6594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:06.602288 6594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:06.603470 6594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:06.605237 6594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:06.617705 6608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:06.617780 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:06.617743 6607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:06.618706 6594 server_base.cc:1034] running on GCE node
I20260809 13:14:06.619307 6594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:06.620213 6594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:06.621423 6594 hybrid_clock.cc:648] HybridClock initialized: now 1786281246621379 us; error 32 us; skew 500 ppm
I20260809 13:14:06.621752 6594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:06.623838 6594 webserver.cc:460] Webserver started at http://127.10.181.129:42153/ using document root <none> and password file <none>
I20260809 13:14:06.624724 6594 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:06.624854 6594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:06.625186 6594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:06.627983 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "ddbd42124afc4569a7f6ddbb97059a5c"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "b253fafe5f0514365769ff82424fb28c"
server_key_iv: "8271bcae583a30352cd52332373ffbff"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.628712 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "ddbd42124afc4569a7f6ddbb97059a5c"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "b253fafe5f0514365769ff82424fb28c"
server_key_iv: "8271bcae583a30352cd52332373ffbff"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.633544 6594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:14:06.636876 6620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:06.638160 6594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:06.638310 6594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "ddbd42124afc4569a7f6ddbb97059a5c"
format_stamp: "Formatted at 2026-08-09 13:14:06 on dist-test-slave-42z9"
server_key: "b253fafe5f0514365769ff82424fb28c"
server_key_iv: "8271bcae583a30352cd52332373ffbff"
server_key_version: "encryptionkey@0"
I20260809 13:14:06.638520 6594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:06.656535 6594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:06.657274 6594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:06.657613 6594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:06.658557 6594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:06.659927 6594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:06.660017 6594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:06.660126 6594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:06.660205 6594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:06.690754 6594 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:42367
I20260809 13:14:06.690788 6806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:42367 every 8 connection(s)
I20260809 13:14:06.692049 6594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:06.696877 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 6594
I20260809 13:14:06.697042 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:06.697507 10966 external_mini_cluster.cc:1428] Setting key 9879d0d4752f3e1c7d43d5a8686598a6
I20260809 13:14:06.700659 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--builtin_ntp_servers=127.10.181.148:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:06.704779 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:06.694508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:49779 (local address 127.10.181.190:34829)
0809 13:14:06.694979 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:06.694985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:06.696108 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 13:14:06.697982 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:06.697988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:06.698088 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:06.698190 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:06.699499 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.700518 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.702117 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.702703 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.702824 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:06.704039 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:06.704045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:06.704049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:06.704069 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:06.704121 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:06.704135 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:06.704213 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:06.704386 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 13:14:06.704485 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":228,"threads_started":1}
I20260809 13:14:06.705715 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:06.695092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34829 (local address 127.10.181.129:49779)
0809 13:14:06.695961 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:06.695995 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:06.697697 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:06.698425 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:06.698433 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:06.698516 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:06.699310 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:06.699338 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.700664 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.700668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:06.701977 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:06.701990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:06.702884 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:06.702889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:06.703079 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:06.703731 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:06.703734 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:06.703778 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:06.703859 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:06.704221 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:06.704228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:06.704233 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:14:06.704357 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 13:14:06.704651 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":249,"threads_started":1}
I20260809 13:14:06.707109 6808 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34829
I20260809 13:14:06.707402 6808 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:06.708103 6808 heartbeater.cc:511] Master 127.10.181.190:34829 requested a full tablet report, sending...
I20260809 13:14:06.710101 6502 ts_manager.cc:194] Registered new tserver with Master: ddbd42124afc4569a7f6ddbb97059a5c (127.10.181.129:42367)
I20260809 13:14:06.711555 6502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.129:49779
W20260809 13:14:07.035048 6813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:07.035301 6813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:07.035400 6813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:07.042428 6813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:07.042552 6813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:07.042711 6813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:07.051407 6813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:07.052542 6813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:07.054312 6813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:07.066649 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:07.066604 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:07.066690 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:07.067731 6813 server_base.cc:1034] running on GCE node
I20260809 13:14:07.068289 6813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:07.069192 6813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:07.070393 6813 hybrid_clock.cc:648] HybridClock initialized: now 1786281247070349 us; error 30 us; skew 500 ppm
I20260809 13:14:07.070698 6813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:07.072834 6813 webserver.cc:460] Webserver started at http://127.10.181.130:32873/ using document root <none> and password file <none>
I20260809 13:14:07.073518 6813 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:07.073634 6813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:07.073930 6813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:07.076694 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "1dd9819df50f488aa79d7f64bb8fde0f"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "7addde04b37e8f289a26aa671144ef22"
server_key_iv: "233a091e6156a557cebf58a00acbd16b"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.077430 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "1dd9819df50f488aa79d7f64bb8fde0f"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "7addde04b37e8f289a26aa671144ef22"
server_key_iv: "233a091e6156a557cebf58a00acbd16b"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.082300 6813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:14:07.085565 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.086764 6813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:14:07.086918 6813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "1dd9819df50f488aa79d7f64bb8fde0f"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "7addde04b37e8f289a26aa671144ef22"
server_key_iv: "233a091e6156a557cebf58a00acbd16b"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.087105 6813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:07.108183 6813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:07.108911 6813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:07.109186 6813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:07.110150 6813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:07.111531 6813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:07.111634 6813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.111737 6813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:07.111807 6813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.141531 6813 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33015
I20260809 13:14:07.141633 7000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33015 every 8 connection(s)
I20260809 13:14:07.142930 6813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:07.151638 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 6813
I20260809 13:14:07.151975 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:07.152844 10966 external_mini_cluster.cc:1428] Setting key 50f7f42e9954a502b00c804d3b6ec508
I20260809 13:14:07.157377 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--builtin_ntp_servers=127.10.181.148:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:07.157716 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.145370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:44219 (local address 127.10.181.190:34829)
0809 13:14:07.145578 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.145583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.147054 (+ 1471us) server_negotiation.cc:407] Connection header received
0809 13:14:07.148818 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.148823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.148909 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.148999 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:07.150162 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.151329 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.153739 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.154353 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.154453 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.156536 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:07.156544 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:07.156549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:07.156573 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:07.156649 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:07.156692 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:07.156798 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.157259 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.157361 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 13:14:07.159597 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.145939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34829 (local address 127.10.181.130:44219)
0809 13:14:07.146894 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.146939 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.148553 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.149244 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.149252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.149334 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:07.149934 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.149950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.151531 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.151537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.153545 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.153562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.154780 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.154788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.155091 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.155946 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:07.155949 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:07.156040 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:07.156220 (+ 180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:07.156904 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:07.156916 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:07.156951 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.157903 (+ 952us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.158301 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":317,"threads_started":1}
I20260809 13:14:07.161118 7003 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34829
I20260809 13:14:07.161473 7003 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:07.162365 7003 heartbeater.cc:511] Master 127.10.181.190:34829 requested a full tablet report, sending...
I20260809 13:14:07.164657 6502 ts_manager.cc:194] Registered new tserver with Master: 1dd9819df50f488aa79d7f64bb8fde0f (127.10.181.130:33015)
I20260809 13:14:07.165548 6502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.130:44219
W20260809 13:14:07.492571 7010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:07.492837 7010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:07.492964 7010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:07.499948 7010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:07.500047 7010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:07.500192 7010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:07.508800 7010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:34829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:07.509994 7010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:07.511771 7010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:07.524238 7018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:07.524135 7019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:07.524202 7021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:07.525029 7010 server_base.cc:1034] running on GCE node
I20260809 13:14:07.525686 7010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:07.526564 7010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:07.527763 7010 hybrid_clock.cc:648] HybridClock initialized: now 1786281247527715 us; error 37 us; skew 500 ppm
I20260809 13:14:07.528055 7010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:07.530177 7010 webserver.cc:460] Webserver started at http://127.10.181.131:35545/ using document root <none> and password file <none>
I20260809 13:14:07.530814 7010 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:07.530916 7010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:07.531224 7010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:07.534006 7010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "89ce790a872c4295ac5e7be690466c8c"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "2dc780eab97e779b79af183806fb87b4"
server_key_iv: "680d2efcc94a52871da7e67a3b87536e"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.534777 7010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "89ce790a872c4295ac5e7be690466c8c"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "2dc780eab97e779b79af183806fb87b4"
server_key_iv: "680d2efcc94a52871da7e67a3b87536e"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.539698 7010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:14:07.543057 7028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.544310 7010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:14:07.544464 7010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "89ce790a872c4295ac5e7be690466c8c"
format_stamp: "Formatted at 2026-08-09 13:14:07 on dist-test-slave-42z9"
server_key: "2dc780eab97e779b79af183806fb87b4"
server_key_iv: "680d2efcc94a52871da7e67a3b87536e"
server_key_version: "encryptionkey@0"
I20260809 13:14:07.544653 7010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:07.563396 7010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:07.564163 7010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:07.564445 7010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:07.565359 7010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:07.566817 7010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:07.566924 7010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.567028 7010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:07.567103 7010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:07.597069 7010 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:40095
I20260809 13:14:07.597142 7186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:40095 every 8 connection(s)
I20260809 13:14:07.598387 7010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:07.608558 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 7010
I20260809 13:14:07.608726 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:07.609297 10966 external_mini_cluster.cc:1428] Setting key 07edaac093545db1538532122cd1ad9e
I20260809 13:14:07.610149 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.600798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:42619 (local address 127.10.181.190:34829)
0809 13:14:07.601012 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.601018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.602378 (+ 1360us) server_negotiation.cc:407] Connection header received
0809 13:14:07.604070 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.604074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.604151 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.604239 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:07.605256 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.606056 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.607510 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.607917 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.608001 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.609288 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:07.609293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:07.609296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:07.609314 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:07.609349 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:07.609361 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:07.609467 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.609755 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.609846 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:14:07.611158 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.601347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:34829 (local address 127.10.181.131:42619)
0809 13:14:07.602224 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.602269 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.603828 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.604406 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.604413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.604493 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:07.605117 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.605136 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.606181 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.606185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.607401 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.607413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.608078 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.608082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.608286 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.608943 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:07.608947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:07.608998 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:07.609097 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:07.609482 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:07.609494 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:07.609502 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.609712 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.610012 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":263,"threads_started":1}
I20260809 13:14:07.612351 7187 heartbeater.cc:347] Connected to a master server at 127.10.181.190:34829
I20260809 13:14:07.612622 7187 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:07.613291 7187 heartbeater.cc:511] Master 127.10.181.190:34829 requested a full tablet report, sending...
I20260809 13:14:07.614629 6502 ts_manager.cc:194] Registered new tserver with Master: 89ce790a872c4295ac5e7be690466c8c (127.10.181.131:40095)
I20260809 13:14:07.615433 6502 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.131:42619
I20260809 13:14:07.624373 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:07.633642 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.628392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.10.181.190:34829)
0809 13:14:07.628545 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.628549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.628998 (+ 449us) server_negotiation.cc:407] Connection header received
0809 13:14:07.629129 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.629133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.629207 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.629283 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:07.629750 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.630417 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.631686 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.632071 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.632154 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.633047 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:07.633052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:07.633055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:07.633073 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:07.633107 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:07.633120 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:07.633183 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.633268 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.633338 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 13:14:07.637017 6502 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:07.643671 6502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:07.692008 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.655596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55742)
0809 13:14:07.683003 (+ 27407us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.683028 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.683855 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.684609 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.684618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.684647 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:07.685087 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.685101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.686969 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.686975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.689220 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.689231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.691023 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.691030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.691251 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.691304 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.691475 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.691667 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27260,"mutex_wait_us":39,"thread_start_us":26867,"threads_started":1}
I20260809 13:14:07.692140 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.656215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.10.181.129:42367)
0809 13:14:07.680514 (+ 24299us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.680523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.683167 (+ 2644us) server_negotiation.cc:407] Connection header received
0809 13:14:07.684058 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.684065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.684379 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.684599 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:07.685326 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.686792 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.689384 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.690840 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.690992 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.691188 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.691573 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.691754 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24097,"thread_start_us":24307,"threads_started":1}
I20260809 13:14:07.693260 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.654677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40095 (local address 127.0.0.1:35948)
0809 13:14:07.681407 (+ 26730us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.681494 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.681727 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.684265 (+ 2538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.684270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.684295 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:07.684705 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.684724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.686505 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.686510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.688387 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.688403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.690373 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.690379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.692683 (+ 2304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.692710 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.692784 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.692900 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26403,"mutex_wait_us":1700,"thread_start_us":28959,"threads_started":1}
I20260809 13:14:07.693356 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.654928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.10.181.131:40095)
0809 13:14:07.683687 (+ 28759us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.683696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.683730 (+ 34us) server_negotiation.cc:407] Connection header received
0809 13:14:07.683826 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.683833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.684065 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.684263 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:07.684949 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.686330 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.688614 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.690135 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.690638 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.690852 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.692842 (+ 1990us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.693009 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28592,"thread_start_us":28449,"threads_started":1}
I20260809 13:14:07.694515 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.655180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.10.181.130:33015)
0809 13:14:07.679902 (+ 24722us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.679914 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.682459 (+ 2545us) server_negotiation.cc:407] Connection header received
0809 13:14:07.683060 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.683069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.683336 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.683508 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:07.684513 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.685949 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.689094 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.691852 (+ 2758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.692040 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.692227 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.694001 (+ 1774us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.694174 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24497,"thread_start_us":24756,"threads_started":1}
I20260809 13:14:07.695241 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.654918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33015 (local address 127.0.0.1:55316)
0809 13:14:07.682331 (+ 27413us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.682353 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.682850 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.683878 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.683885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.683912 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:07.684269 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.684282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.686143 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.686148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.688896 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.688911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.692344 (+ 3433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.692351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.692530 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.692560 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.694765 (+ 2205us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.694898 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27287,"mutex_wait_us":229,"thread_start_us":27191,"threads_started":1}
I20260809 13:14:07.696584 6699 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd (DEFAULT_TABLE table=test-table [id=1c3a0d9c66db4b79933a2ed5376650bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:07.697922 6699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc945f439c14b129d77b8cf86bf12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:07.698971 7094 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd (DEFAULT_TABLE table=test-table [id=1c3a0d9c66db4b79933a2ed5376650bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:07.700436 7094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc945f439c14b129d77b8cf86bf12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:07.700662 6897 tablet_service.cc:1467] Processing CreateTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd (DEFAULT_TABLE table=test-table [id=1c3a0d9c66db4b79933a2ed5376650bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:07.701988 6897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fc945f439c14b129d77b8cf86bf12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:07.709035 7214 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Bootstrap starting.
I20260809 13:14:07.712164 7214 tablet_bootstrap.cc:654] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:07.713129 7214 log.cc:824] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:07.714265 6808 heartbeater.cc:503] Master 127.10.181.190:34829 was elected leader, sending a full tablet report...
I20260809 13:14:07.716212 7216 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Bootstrap starting.
I20260809 13:14:07.719554 7214 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: No bootstrap required, opened a new log
I20260809 13:14:07.719841 7214 ts_tablet_manager.cc:1397] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260809 13:14:07.720120 7216 tablet_bootstrap.cc:654] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:07.720424 7217 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Bootstrap starting.
I20260809 13:14:07.721608 7216 log.cc:824] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:07.724226 7216 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: No bootstrap required, opened a new log
I20260809 13:14:07.724270 7217 tablet_bootstrap.cc:654] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:07.724469 7216 ts_tablet_manager.cc:1397] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 13:14:07.725477 7217 log.cc:824] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:07.728498 7217 tablet_bootstrap.cc:492] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: No bootstrap required, opened a new log
I20260809 13:14:07.728766 7217 ts_tablet_manager.cc:1397] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:14:07.729677 7214 raft_consensus.cc:348] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.730137 7214 raft_consensus.cc:374] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:07.730253 7214 raft_consensus.cc:729] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddbd42124afc4569a7f6ddbb97059a5c, State: Initialized, Role: FOLLOWER
I20260809 13:14:07.730906 7214 consensus_queue.cc:260] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.731966 7216 raft_consensus.cc:348] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.732402 7216 raft_consensus.cc:374] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:07.732499 7216 raft_consensus.cc:729] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd9819df50f488aa79d7f64bb8fde0f, State: Initialized, Role: FOLLOWER
I20260809 13:14:07.733035 7216 consensus_queue.cc:260] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.734140 7003 heartbeater.cc:503] Master 127.10.181.190:34829 was elected leader, sending a full tablet report...
I20260809 13:14:07.734570 7216 ts_tablet_manager.cc:1428] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260809 13:14:07.736125 7214 ts_tablet_manager.cc:1428] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260809 13:14:07.740052 7217 raft_consensus.cc:348] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.740365 7217 raft_consensus.cc:374] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:07.740456 7217 raft_consensus.cc:729] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ce790a872c4295ac5e7be690466c8c, State: Initialized, Role: FOLLOWER
I20260809 13:14:07.740936 7217 consensus_queue.cc:260] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.741896 7187 heartbeater.cc:503] Master 127.10.181.190:34829 was elected leader, sending a full tablet report...
I20260809 13:14:07.742444 7217 ts_tablet_manager.cc:1428] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 13:14:07.836575 7221 raft_consensus.cc:482] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:07.836941 7221 raft_consensus.cc:504] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.838837 7221 leader_election.cc:288] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ce790a872c4295ac5e7be690466c8c (127.10.181.131:40095), ddbd42124afc4569a7f6ddbb97059a5c (127.10.181.129:42367)
I20260809 13:14:07.847601 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.839517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:60297 (local address 127.10.181.129:42367)
0809 13:14:07.839740 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.839747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.840372 (+ 625us) server_negotiation.cc:407] Connection header received
0809 13:14:07.840522 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.840526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.840625 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.840721 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:07.841393 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.842436 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.845350 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.846559 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.846733 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.846805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.847227 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.847356 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 13:14:07.847601 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.839298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:37299 (local address 127.10.181.131:40095)
0809 13:14:07.839544 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.839551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.840270 (+ 719us) server_negotiation.cc:407] Connection header received
0809 13:14:07.840441 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.840448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.840526 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.840592 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:07.841243 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.842401 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.845350 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.846566 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.846736 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.846805 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.847227 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.847356 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260809 13:14:07.847761 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.839286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:40095 (local address 127.10.181.130:37299)
0809 13:14:07.840145 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.840165 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.840298 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.840636 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.840641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.840673 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:07.841080 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.841094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.842535 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.842541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.845159 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.845169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.846770 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.846777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.846999 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.847096 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.847173 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.847305 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":226,"thread_start_us":445,"threads_started":1}
I20260809 13:14:07.847783 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.839335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.10.181.130:60297)
0809 13:14:07.840240 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:07.840257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:07.840396 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:07.840752 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:07.840755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:07.840789 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:07.841215 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.841226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.842600 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.842609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.845129 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:07.845148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.846778 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.846784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:07.846994 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.847084 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 13:14:07.847173 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:14:07.847326 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":206,"spinlock_wait_cycles":20736,"thread_start_us":571,"threads_started":1}
I20260809 13:14:07.848860 7125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc945f439c14b129d77b8cf86bf12cd" candidate_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ce790a872c4295ac5e7be690466c8c" is_pre_election: true
I20260809 13:14:07.849162 6735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc945f439c14b129d77b8cf86bf12cd" candidate_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" is_pre_election: true
I20260809 13:14:07.849529 7125 raft_consensus.cc:2454] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dd9819df50f488aa79d7f64bb8fde0f in term 0.
I20260809 13:14:07.849748 6735 raft_consensus.cc:2454] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dd9819df50f488aa79d7f64bb8fde0f in term 0.
W20260809 13:14:07.849923 7188 tablet.cc:2369] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:07.850380 6832 leader_election.cc:302] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd9819df50f488aa79d7f64bb8fde0f, 89ce790a872c4295ac5e7be690466c8c; no voters:
I20260809 13:14:07.850795 7221 raft_consensus.cc:2781] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:07.850929 7221 raft_consensus.cc:482] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:07.851033 7221 raft_consensus.cc:3037] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:07.853223 7221 raft_consensus.cc:504] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.853790 7221 leader_election.cc:288] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [CANDIDATE]: Term 1 election: Requested vote from peers 89ce790a872c4295ac5e7be690466c8c (127.10.181.131:40095), ddbd42124afc4569a7f6ddbb97059a5c (127.10.181.129:42367)
I20260809 13:14:07.854228 6735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc945f439c14b129d77b8cf86bf12cd" candidate_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbd42124afc4569a7f6ddbb97059a5c"
I20260809 13:14:07.854225 7125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fc945f439c14b129d77b8cf86bf12cd" candidate_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ce790a872c4295ac5e7be690466c8c"
I20260809 13:14:07.854390 6735 raft_consensus.cc:3037] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:07.854406 7125 raft_consensus.cc:3037] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:07.856500 6735 raft_consensus.cc:2454] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dd9819df50f488aa79d7f64bb8fde0f in term 1.
I20260809 13:14:07.856577 7125 raft_consensus.cc:2454] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dd9819df50f488aa79d7f64bb8fde0f in term 1.
I20260809 13:14:07.856860 6838 leader_election.cc:302] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd9819df50f488aa79d7f64bb8fde0f, ddbd42124afc4569a7f6ddbb97059a5c; no voters:
I20260809 13:14:07.857167 7221 raft_consensus.cc:2781] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:07.857612 7221 raft_consensus.cc:686] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 1 LEADER]: Becoming Leader. State: Replica: 1dd9819df50f488aa79d7f64bb8fde0f, State: Running, Role: LEADER
I20260809 13:14:07.858043 7221 consensus_queue.cc:237] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } }
I20260809 13:14:07.862658 6502 catalog_manager.cc:5495] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dd9819df50f488aa79d7f64bb8fde0f (127.10.181.130). New cstate: current_term: 1 leader_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd9819df50f488aa79d7f64bb8fde0f" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 } health_report { overall_health: UNKNOWN } } }
I20260809 13:14:07.893225 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.887772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55328 (local address 127.10.181.130:33015)
0809 13:14:07.887968 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.887975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.888050 (+ 75us) server_negotiation.cc:407] Connection header received
0809 13:14:07.888286 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.888291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.888382 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.888514 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:07.888907 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.889605 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.891082 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.891577 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.891687 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.892159 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:07.892648 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:07.892786 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.892887 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.892963 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 13:14:07.894802 7005 tablet.cc:2369] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:07.901754 6735 raft_consensus.cc:1261] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 1 FOLLOWER]: Refusing update from remote peer 1dd9819df50f488aa79d7f64bb8fde0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:07.901753 7125 raft_consensus.cc:1261] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 1 FOLLOWER]: Refusing update from remote peer 1dd9819df50f488aa79d7f64bb8fde0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:07.902691 7225 consensus_queue.cc:1035] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddbd42124afc4569a7f6ddbb97059a5c" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 42367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:07.902992 7221 consensus_queue.cc:1035] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ce790a872c4295ac5e7be690466c8c" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 40095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:07.913012 7230 mvcc.cc:205] Tried to move back new op lower bound from 7316607991395016704 to 7316607991230091264. Current Snapshot: MvccSnapshot[applied={T|T < 7316607991395016704}]
I20260809 13:14:07.916891 7231 mvcc.cc:205] Tried to move back new op lower bound from 7316607991395016704 to 7316607991230091264. Current Snapshot: MvccSnapshot[applied={T|T < 7316607991395016704}]
I20260809 13:14:07.932762 6502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:07.932984 6502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:07.936702 6502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4933fc14629f490ab96fca80922a234d: Sending DeleteTablet for 3 replicas of tablet 8fc945f439c14b129d77b8cf86bf12cd
I20260809 13:14:07.937850 7094 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:07 UTC) from {username='slave'} at 127.0.0.1:35948
I20260809 13:14:07.937935 6897 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:07 UTC) from {username='slave'} at 127.0.0.1:55316
I20260809 13:14:07.938678 7244 tablet_replica.cc:330] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: stopping tablet replica
I20260809 13:14:07.938669 6699 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fc945f439c14b129d77b8cf86bf12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:07 UTC) from {username='slave'} at 127.0.0.1:55742
I20260809 13:14:07.938969 7244 raft_consensus.cc:2229] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:07.939432 7244 raft_consensus.cc:2258] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:07.939541 7246 tablet_replica.cc:330] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: stopping tablet replica
I20260809 13:14:07.939958 7246 raft_consensus.cc:2229] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:07.940483 7246 raft_consensus.cc:2258] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:07.940768 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 6594
I20260809 13:14:07.941748 7244 ts_tablet_manager.cc:1905] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:07.949234 7244 ts_tablet_manager.cc:1918] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:07.949461 7244 log.cc:1196] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/8fc945f439c14b129d77b8cf86bf12cd
I20260809 13:14:07.950023 7244 ts_tablet_manager.cc:1939] T 8fc945f439c14b129d77b8cf86bf12cd P 89ce790a872c4295ac5e7be690466c8c: Deleting consensus metadata
I20260809 13:14:07.951542 7246 ts_tablet_manager.cc:1905] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:07.951979 6477 catalog_manager.cc:4841] TS 89ce790a872c4295ac5e7be690466c8c (127.10.181.131:40095): tablet 8fc945f439c14b129d77b8cf86bf12cd (table test-table [id=1c3a0d9c66db4b79933a2ed5376650bc]) successfully deleted
I20260809 13:14:07.953974 7245 tablet_replica.cc:330] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: stopping tablet replica
I20260809 13:14:07.954510 7245 raft_consensus.cc:2229] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:07.955273 7245 raft_consensus.cc:2258] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:07.957695 7245 ts_tablet_manager.cc:1905] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:07.961352 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:07.941666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.10.181.129:42367)
0809 13:14:07.941856 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:07.941866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:07.953414 (+ 11548us) server_negotiation.cc:407] Connection header received
0809 13:14:07.953853 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:07.953859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:07.953962 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:07.954044 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:07.954752 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.955909 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.957171 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:07.957852 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:07.957972 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:07.959372 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:07.959401 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:07.959422 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:07.960195 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:07.960250 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:07.960273 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:07.960365 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:07.960871 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 13:14:07.960973 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:14:07.961512 7246 ts_tablet_manager.cc:1918] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:07.962018 7246 log.cc:1196] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/8fc945f439c14b129d77b8cf86bf12cd
I20260809 13:14:07.962450 6801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:07.965720 7245 ts_tablet_manager.cc:1918] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:07.965999 7245 log.cc:1196] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/8fc945f439c14b129d77b8cf86bf12cd
I20260809 13:14:07.966553 7245 ts_tablet_manager.cc:1939] T 8fc945f439c14b129d77b8cf86bf12cd P 1dd9819df50f488aa79d7f64bb8fde0f: Deleting consensus metadata
I20260809 13:14:07.968233 6479 catalog_manager.cc:4841] TS 1dd9819df50f488aa79d7f64bb8fde0f (127.10.181.130:33015): tablet 8fc945f439c14b129d77b8cf86bf12cd (table test-table [id=1c3a0d9c66db4b79933a2ed5376650bc]) successfully deleted
I20260809 13:14:08.071702 7246 ts_tablet_manager.cc:1939] T 8fc945f439c14b129d77b8cf86bf12cd P ddbd42124afc4569a7f6ddbb97059a5c: Deleting consensus metadata
I20260809 13:14:08.072495 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 6594
W20260809 13:14:08.096793 6480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:42367 (error 108)
I20260809 13:14:08.096935 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 6813
I20260809 13:14:08.099154 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.098612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55752)
0809 13:14:08.098861 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.098980 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260809 13:14:08.099412 6480 catalog_manager.cc:4584] TS ddbd42124afc4569a7f6ddbb97059a5c (127.10.181.129:42367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fc945f439c14b129d77b8cf86bf12cd: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
I20260809 13:14:08.104411 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.097889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.10.181.130:33015)
0809 13:14:08.098141 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:08.098147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:08.098205 (+ 58us) server_negotiation.cc:407] Connection header received
0809 13:14:08.098422 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:08.098426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:08.098512 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:08.098615 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:08.099084 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:08.099956 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:08.101276 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:08.101815 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:08.101941 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:08.103015 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:08.103024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:08.103039 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:08.103659 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:08.103708 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:08.103731 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:08.103834 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:08.103982 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 13:14:08.104091 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 13:14:08.105108 6995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:08.153223 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.152659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55754)
0809 13:14:08.152907 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.153029 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 13:14:08.154419 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.154000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55768)
0809 13:14:08.154147 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.154238 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 13:14:08.215238 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.214775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55770)
0809 13:14:08.214987 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.215084 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 13:14:08.216303 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.215923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55776)
0809 13:14:08.216085 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.216170 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 13:14:08.218822 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 6813
I20260809 13:14:08.245899 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 7010
I20260809 13:14:08.252388 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.246712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.10.181.131:40095)
0809 13:14:08.246898 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:08.246903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:08.246922 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:14:08.247087 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:08.247091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:08.247184 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:08.247269 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:08.247622 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:08.248312 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:08.249772 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:08.250151 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:08.250242 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:08.251148 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:08.251158 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:08.251173 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:08.251752 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:08.251802 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:08.251823 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:08.251915 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:08.251985 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 13:14:08.252071 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:14:08.253038 7178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:08.296814 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.296311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55792)
0809 13:14:08.296529 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.296635 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 13:14:08.297955 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.297545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55800)
0809 13:14:08.297749 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.297828 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 13:14:08.340770 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 7010
I20260809 13:14:08.363382 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 6450
I20260809 13:14:08.364480 6551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:08.472433 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 6450
I20260809 13:14:08.472500 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:08.472022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:42367 (local address 127.0.0.1:55812)
0809 13:14:08.472244 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:08.472339 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:42367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
2026-08-09T13:14:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2855 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:08 dist-test-slave-42z9 krb5kdc[7257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:08 dist-test-slave-42z9 krb5kdc[7257](info): set up 2 sockets
Aug 09 13:14:08 dist-test-slave-42z9 krb5kdc[7257](info): commencing operation
krb5kdc: starting...
W20260809 13:14:09.716410 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:09 dist-test-slave-42z9 krb5kdc[7257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:09.998018 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40135
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:40135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:10.331348 7277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:10.331629 7277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:10.331758 7277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:10.338712 7277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:10.338816 7277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:10.338889 7277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:10.338973 7277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:10.339026 7277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:10.349987 7277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:40135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:10.351640 7277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:10.353219 7277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:10.364212 7288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:10.365100 7289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:10.366211 7292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:10.366529 7277 server_base.cc:1034] running on GCE node
I20260809 13:14:10.367270 7277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:10.368611 7277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:10.369812 7277 hybrid_clock.cc:648] HybridClock initialized: now 1786281250369765 us; error 30 us; skew 500 ppm
I20260809 13:14:10.370144 7277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:10 dist-test-slave-42z9 krb5kdc[7257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281250, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:10.373101 7277 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:10.374575 7277 webserver.cc:460] Webserver started at http://127.10.181.190:40243/ using document root <none> and password file <none>
I20260809 13:14:10.375252 7277 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:10.375367 7277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:10.375674 7277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:10.378700 7277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "f802cab36f114951930c18c28d07a9d0"
format_stamp: "Formatted at 2026-08-09 13:14:10 on dist-test-slave-42z9"
server_key: "1f34644e67f0ba45a11061ce83b902e0"
server_key_iv: "347fc45753bb3ed38a47744462981869"
server_key_version: "encryptionkey@0"
I20260809 13:14:10.379420 7277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "f802cab36f114951930c18c28d07a9d0"
format_stamp: "Formatted at 2026-08-09 13:14:10 on dist-test-slave-42z9"
server_key: "1f34644e67f0ba45a11061ce83b902e0"
server_key_iv: "347fc45753bb3ed38a47744462981869"
server_key_version: "encryptionkey@0"
I20260809 13:14:10.384399 7277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:14:10.387760 7301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:10.388993 7277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:14:10.389151 7277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "f802cab36f114951930c18c28d07a9d0"
format_stamp: "Formatted at 2026-08-09 13:14:10 on dist-test-slave-42z9"
server_key: "1f34644e67f0ba45a11061ce83b902e0"
server_key_iv: "347fc45753bb3ed38a47744462981869"
server_key_version: "encryptionkey@0"
I20260809 13:14:10.389318 7277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:10.403738 7277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:10.407658 7277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:10.407908 7277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:10.424945 7277 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:40135
I20260809 13:14:10.424950 7368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:40135 every 8 connection(s)
I20260809 13:14:10.426244 7277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:10.427222 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 7277
I20260809 13:14:10.427423 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:10.427948 10966 external_mini_cluster.cc:1428] Setting key 351e4e644dda906f8b3a4be4a99328ca
I20260809 13:14:10.430759 7369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:10.441097 7369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: Bootstrap starting.
Aug 09 13:14:10 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:10.444929 7369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:10.446290 7369 log.cc:824] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:10.449594 7369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: No bootstrap required, opened a new log
I20260809 13:14:10.450641 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:10.429860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.10.181.190:40135)
0809 13:14:10.430606 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:10.430617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:10.430658 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:10.432325 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:10.432347 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:10.432682 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:10.433026 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:10.434581 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:10.435928 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:10.437254 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:10.437946 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:10.438133 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:10.442352 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:10.442363 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:10.442380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:10.442431 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:10.446255 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:10.447027 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:10.447033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:10.447039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:10.447217 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:10.447666 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:10.447670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:10.447674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:10.448347 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:10.448644 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:10.448932 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 13:14:10.449353 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":346,"threads_started":1}
I20260809 13:14:10.457139 7369 raft_consensus.cc:348] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } }
I20260809 13:14:10.457481 7369 raft_consensus.cc:374] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:10.457578 7369 raft_consensus.cc:729] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f802cab36f114951930c18c28d07a9d0, State: Initialized, Role: FOLLOWER
I20260809 13:14:10.458081 7369 consensus_queue.cc:260] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } }
I20260809 13:14:10.458289 7369 raft_consensus.cc:388] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:10.458387 7369 raft_consensus.cc:482] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:10.458515 7369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:10.460536 7369 raft_consensus.cc:504] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } }
I20260809 13:14:10.460984 7369 leader_election.cc:302] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f802cab36f114951930c18c28d07a9d0; no voters:
I20260809 13:14:10.461596 7369 leader_election.cc:288] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:10.461706 7375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:10.462713 7375 raft_consensus.cc:686] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [term 1 LEADER]: Becoming Leader. State: Replica: f802cab36f114951930c18c28d07a9d0, State: Running, Role: LEADER
I20260809 13:14:10.463200 7375 consensus_queue.cc:237] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } }
I20260809 13:14:10.463603 7369 sys_catalog.cc:564] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:10.466086 7379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f802cab36f114951930c18c28d07a9d0. Latest consensus state: current_term: 1 leader_uuid: "f802cab36f114951930c18c28d07a9d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } } }
I20260809 13:14:10.466480 7379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:10.467429 7377 sys_catalog.cc:455] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f802cab36f114951930c18c28d07a9d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f802cab36f114951930c18c28d07a9d0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40135 } } }
I20260809 13:14:10.468065 7377 sys_catalog.cc:458] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:10.469240 7386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:10.473237 7386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:10.481318 7386 catalog_manager.cc:1347] Generated new cluster ID: af92cf5d08a8480e834e1a4db9c47ca1
I20260809 13:14:10.481428 7386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:10.500972 7386 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:10.502354 7386 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:10.511673 7386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: Generated new TSK 0
I20260809 13:14:10.512501 7386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:14:10.592022 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:10.989744 7410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:10.990020 7410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:10.990167 7410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:10.997462 7410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:10.997583 7410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:10.997752 7410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:11.007359 7410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:11.008673 7410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:11.010038 7410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:11.023044 7417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:11.023113 7418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:11.023078 7420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:11.024261 7410 server_base.cc:1034] running on GCE node
I20260809 13:14:11.024868 7410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:11.025815 7410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:11.027017 7410 hybrid_clock.cc:648] HybridClock initialized: now 1786281251026973 us; error 36 us; skew 500 ppm
I20260809 13:14:11.027324 7410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:11 dist-test-slave-42z9 krb5kdc[7257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281251, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:11.030383 7410 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:14:11.031855 7410 webserver.cc:460] Webserver started at http://127.10.181.129:39207/ using document root <none> and password file <none>
I20260809 13:14:11.032523 7410 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:11.032642 7410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:11.032943 7410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:11.035897 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "0ae354f35f5e48748370544e73e24098"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "b949912e6458917a6d6b720080008f48"
server_key_iv: "761adb229b7615dcd7fed0f077243ffb"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.036664 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "0ae354f35f5e48748370544e73e24098"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "b949912e6458917a6d6b720080008f48"
server_key_iv: "761adb229b7615dcd7fed0f077243ffb"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.041814 7410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 13:14:11.045331 7430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.046619 7410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:11.046766 7410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "0ae354f35f5e48748370544e73e24098"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "b949912e6458917a6d6b720080008f48"
server_key_iv: "761adb229b7615dcd7fed0f077243ffb"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.046944 7410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:11.069495 7410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:11.073315 7410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:11.073608 7410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:11.074604 7410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:11.075922 7410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:11.076005 7410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.076125 7410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:11.076179 7410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.107133 7410 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:45865
I20260809 13:14:11.107203 7606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:45865 every 8 connection(s)
I20260809 13:14:11.108436 7410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:11.114743 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 7410
I20260809 13:14:11.114908 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:11.115363 10966 external_mini_cluster.cc:1428] Setting key 9363bb044e72bb504741582aaa2aa562
Aug 09 13:14:11 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281251, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:11.134469 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:11.110967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:39703 (local address 127.10.181.190:40135)
0809 13:14:11.111441 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:11.111447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:11.112403 (+ 956us) server_negotiation.cc:407] Connection header received
0809 13:14:11.114271 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:11.114275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:11.114352 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:11.114480 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:11.116086 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.116857 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.118433 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.118839 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.118956 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:11.128505 (+ 9549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:11.128511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:11.128515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:11.128552 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:11.131261 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:11.132034 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:11.132040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:11.132043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:11.132198 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:11.132610 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:11.132615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:11.132618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:11.132964 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:11.133110 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:11.133865 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 13:14:11.134093 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":216,"threads_started":1}
I20260809 13:14:11.135263 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:11.111515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40135 (local address 127.10.181.129:39703)
0809 13:14:11.112243 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:11.112288 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:11.114017 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:11.114652 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:11.114660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:11.115273 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:11.115879 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:11.115908 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.117027 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.117031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:11.118291 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:11.118305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.119078 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.119086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:11.119303 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:11.125937 (+ 6634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:11.125965 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:11.128246 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:11.131423 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:11.131430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:11.131445 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:11.131894 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:11.132301 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:11.132304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:11.132306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:11.132491 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:11.133077 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:11.133083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:11.133422 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 13:14:11.133709 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 13:14:11.134129 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":270,"threads_started":1}
I20260809 13:14:11.136888 7607 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40135
I20260809 13:14:11.137199 7607 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:11.137962 7607 heartbeater.cc:511] Master 127.10.181.190:40135 requested a full tablet report, sending...
I20260809 13:14:11.140131 7322 ts_manager.cc:194] Registered new tserver with Master: 0ae354f35f5e48748370544e73e24098 (127.10.181.129:45865)
I20260809 13:14:11.141624 7322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:39703
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:14:11.186493 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:11.526816 7615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:11.527081 7615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:11.527218 7615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:11.534209 7615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:11.534309 7615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:11.534462 7615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:11.543825 7615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:11.545027 7615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:11.546316 7615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:11.558624 7624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:11.558681 7625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:11.558723 7628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:11.559470 7615 server_base.cc:1034] running on GCE node
I20260809 13:14:11.560031 7615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:11.560883 7615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:11.562098 7615 hybrid_clock.cc:648] HybridClock initialized: now 1786281251562040 us; error 37 us; skew 500 ppm
I20260809 13:14:11.562386 7615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:11 dist-test-slave-42z9 krb5kdc[7257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281251, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:11.565346 7615 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:14:11.566653 7615 webserver.cc:460] Webserver started at http://127.10.181.130:42261/ using document root <none> and password file <none>
I20260809 13:14:11.567281 7615 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:11.567396 7615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:11.567708 7615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:11.570508 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "9ef59e39e4c94a89803b1f159edf09b4"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "0745a93b55c7f24cb0aff882bb9e640a"
server_key_iv: "37aaa530f7327b44d0dc1a3943da724d"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.571231 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "9ef59e39e4c94a89803b1f159edf09b4"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "0745a93b55c7f24cb0aff882bb9e640a"
server_key_iv: "37aaa530f7327b44d0dc1a3943da724d"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.576081 7615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:14:11.579345 7639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.580518 7615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 13:14:11.580669 7615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "9ef59e39e4c94a89803b1f159edf09b4"
format_stamp: "Formatted at 2026-08-09 13:14:11 on dist-test-slave-42z9"
server_key: "0745a93b55c7f24cb0aff882bb9e640a"
server_key_iv: "37aaa530f7327b44d0dc1a3943da724d"
server_key_version: "encryptionkey@0"
I20260809 13:14:11.580849 7615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:11.593900 7615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:11.597306 7615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:11.597584 7615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:11.598431 7615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:11.599712 7615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:11.599788 7615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.599898 7615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:11.599956 7615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:11.629317 7615 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:33519
I20260809 13:14:11.629364 7809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:33519 every 8 connection(s)
I20260809 13:14:11.630849 7615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:11.635774 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 7615
I20260809 13:14:11.635977 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:11.636529 10966 external_mini_cluster.cc:1428] Setting key 2d6f83117fedd8669a85d2a891b44e20
Aug 09 13:14:11 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281251, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:11.656527 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:11.633425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:45559 (local address 127.10.181.190:40135)
0809 13:14:11.634525 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:11.634532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:11.635069 (+ 537us) server_negotiation.cc:407] Connection header received
0809 13:14:11.637952 (+ 2883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:11.637960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:11.638065 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:11.638150 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:11.640989 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.641972 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.644138 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.644613 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.644727 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:11.648797 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:11.648804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:11.648808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:11.648851 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:11.653034 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:11.653913 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:11.653919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:11.653922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:11.654073 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:11.654566 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:11.654568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:11.654571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:11.654840 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:11.654958 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:11.655898 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 13:14:11.656155 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":928}
I20260809 13:14:11.657595 7813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:11.634027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40135 (local address 127.10.181.130:45559)
0809 13:14:11.634903 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:11.634951 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:11.637551 (+ 2600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:11.638795 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:11.638809 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:11.639745 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:11.640767 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:11.640801 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.642150 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.642156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:11.643981 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:11.643994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:11.644838 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:11.644846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:11.645066 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:11.645720 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:11.645743 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:11.648533 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:11.653174 (+ 4641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:11.653183 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:11.653199 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:11.653777 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:11.654195 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:11.654201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:11.654204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:11.654458 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:11.654967 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:11.654975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:11.655431 (+ 456us) client_negotiation.cc:769] Sending connection context
0809 13:14:11.655737 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 13:14:11.656243 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":392,"threads_started":1}
I20260809 13:14:11.658955 7810 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40135
I20260809 13:14:11.659272 7810 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:11.660111 7810 heartbeater.cc:511] Master 127.10.181.190:40135 requested a full tablet report, sending...
I20260809 13:14:11.664189 7322 ts_manager.cc:194] Registered new tserver with Master: 9ef59e39e4c94a89803b1f159edf09b4 (127.10.181.130:33519)
I20260809 13:14:11.665302 7322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:45559
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:14:11.713327 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:12.041828 7819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:12.042085 7819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:12.042177 7819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:12.049185 7819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:12.049283 7819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:12.049405 7819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:12.058645 7819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:12.060015 7819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:12.061265 7819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:12.072992 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:12.073552 7830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:12.074458 7833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:12.074896 7819 server_base.cc:1034] running on GCE node
I20260809 13:14:12.075481 7819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:12.076339 7819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:12.077540 7819 hybrid_clock.cc:648] HybridClock initialized: now 1786281252077484 us; error 40 us; skew 500 ppm
I20260809 13:14:12.077837 7819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:12 dist-test-slave-42z9 krb5kdc[7257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281252, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:12.080916 7819 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:14:12.082331 7819 webserver.cc:460] Webserver started at http://127.10.181.131:36977/ using document root <none> and password file <none>
I20260809 13:14:12.083000 7819 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:12.083098 7819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:12.083396 7819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:12.086426 7819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "a5e83cb8c59b4065beb39cf6ef1e706f"
format_stamp: "Formatted at 2026-08-09 13:14:12 on dist-test-slave-42z9"
server_key: "adcb2650adffd56afd87f04420c62d40"
server_key_iv: "72c2713295dc0ae72ba9ff6cc0910f78"
server_key_version: "encryptionkey@0"
I20260809 13:14:12.087132 7819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "a5e83cb8c59b4065beb39cf6ef1e706f"
format_stamp: "Formatted at 2026-08-09 13:14:12 on dist-test-slave-42z9"
server_key: "adcb2650adffd56afd87f04420c62d40"
server_key_iv: "72c2713295dc0ae72ba9ff6cc0910f78"
server_key_version: "encryptionkey@0"
I20260809 13:14:12.092171 7819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:14:12.095713 7841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:12.096937 7819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:14:12.097101 7819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "a5e83cb8c59b4065beb39cf6ef1e706f"
format_stamp: "Formatted at 2026-08-09 13:14:12 on dist-test-slave-42z9"
server_key: "adcb2650adffd56afd87f04420c62d40"
server_key_iv: "72c2713295dc0ae72ba9ff6cc0910f78"
server_key_version: "encryptionkey@0"
I20260809 13:14:12.097271 7819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:12.116163 7819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:12.120165 7819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:12.120426 7819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:12.121328 7819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:12.122654 7819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:12.122732 7819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:12.122853 7819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:12.122905 7819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:12.145328 7607 heartbeater.cc:503] Master 127.10.181.190:40135 was elected leader, sending a full tablet report...
I20260809 13:14:12.152818 7819 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:43407
I20260809 13:14:12.152856 8008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:43407 every 8 connection(s)
I20260809 13:14:12.154160 7819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:12.158543 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 7819
I20260809 13:14:12.158715 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:12.159123 10966 external_mini_cluster.cc:1428] Setting key 87e10c7a87d5ff40d7adda6e0aec076a
Aug 09 13:14:12 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281252, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:12.173092 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.156537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:53261 (local address 127.10.181.190:40135)
0809 13:14:12.156742 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.156748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.157939 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 13:14:12.159850 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.159855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.159973 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.160060 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:12.161805 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.162653 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.164268 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.164728 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.164826 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.167971 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:12.167976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:12.167980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:12.168031 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:12.170482 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.171113 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.171126 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.171129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.171251 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.171611 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.171614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.171616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.171854 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:12.171961 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.172567 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.172783 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:14:12.174150 8016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.157034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40135 (local address 127.10.181.131:53261)
0809 13:14:12.157797 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.157836 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.159606 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.160306 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.160313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.160840 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:12.161615 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.161642 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.162784 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.162788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.164107 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.164120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.164917 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.164922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.165121 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.165755 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:12.165782 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:12.167795 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:12.170610 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:12.170616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:12.170626 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:12.171023 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:12.171329 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:12.171332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:12.171334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:12.171529 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:12.171941 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:12.171947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:12.172213 (+ 266us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.172444 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.172891 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":264,"threads_started":1}
I20260809 13:14:12.175357 8009 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40135
I20260809 13:14:12.175616 8009 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:12.176263 8009 heartbeater.cc:511] Master 127.10.181.190:40135 requested a full tablet report, sending...
I20260809 13:14:12.177627 7322 ts_manager.cc:194] Registered new tserver with Master: a5e83cb8c59b4065beb39cf6ef1e706f (127.10.181.131:43407)
I20260809 13:14:12.178392 7322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:53261
I20260809 13:14:12.186318 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:12.202267 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.190274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.10.181.190:40135)
0809 13:14:12.190423 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.190428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.190825 (+ 397us) server_negotiation.cc:407] Connection header received
0809 13:14:12.190986 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.190990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.191051 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.191129 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:12.192655 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.193400 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.194728 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.195113 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.195204 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.197676 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:12.197682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:12.197686 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:12.197715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:12.199898 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.200537 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.200541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.200543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.200687 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.201028 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.201031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.201033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.201347 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:12.201567 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.201914 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.202038 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 13:14:12.205811 7322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:12.212374 7322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:12.246614 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.224749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.10.181.131:43407)
0809 13:14:12.235619 (+ 10870us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.235630 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.236474 (+ 844us) server_negotiation.cc:407] Connection header received
0809 13:14:12.237411 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.237419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.237705 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.237875 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:12.239058 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.240804 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.243745 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.245152 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.245331 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.245553 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.246131 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.246293 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10654,"thread_start_us":10883,"threads_started":1}
I20260809 13:14:12.246714 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.224188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:43407 (local address 127.0.0.1:57440)
0809 13:14:12.236311 (+ 12123us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.236346 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.237117 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.238246 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.238255 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.238290 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:12.238814 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.238828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.241012 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.241018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.243574 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.243589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.245728 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.245735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.245902 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.245937 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.246061 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.246255 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11906,"mutex_wait_us":2,"thread_start_us":11762,"threads_started":1}
I20260809 13:14:12.247211 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.224237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.10.181.130:33519)
0809 13:14:12.236910 (+ 12673us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.236918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.237526 (+ 608us) server_negotiation.cc:407] Connection header received
0809 13:14:12.237746 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.237754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.237994 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.238196 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:12.239268 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.240933 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.244372 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.245656 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.245816 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.245977 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.246525 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.246696 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12515,"thread_start_us":12394,"threads_started":1}
I20260809 13:14:12.248025 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.224534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33519 (local address 127.0.0.1:33284)
0809 13:14:12.237366 (+ 12832us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.237382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.237528 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.238472 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.238478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.238508 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:12.239040 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.239053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.241132 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.241138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.244229 (+ 3091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.244242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.246137 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.246144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.246310 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.246343 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.247538 (+ 1195us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.247668 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12718,"mutex_wait_us":748,"thread_start_us":12566,"threads_started":1}
I20260809 13:14:12.249593 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.237699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46140)
0809 13:14:12.241642 (+ 3943us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.241659 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.241770 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.242733 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.242737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.242762 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:12.243109 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.243122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.244810 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.244815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.247252 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.247268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.248868 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.248875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.249032 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.249062 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.249144 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.249261 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3805,"thread_start_us":3636,"threads_started":1}
I20260809 13:14:12.250903 7911 tablet_service.cc:1467] Processing CreateTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e (DEFAULT_TABLE table=test-table [id=f602dac8400d4cb7b160d27b799d789d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:12.251495 7712 tablet_service.cc:1467] Processing CreateTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e (DEFAULT_TABLE table=test-table [id=f602dac8400d4cb7b160d27b799d789d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:12.252264 7911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bd50b337f14d34a9b47b8455c31e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:12.252589 7712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bd50b337f14d34a9b47b8455c31e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:12.258199 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.237887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.10.181.129:45865)
0809 13:14:12.241999 (+ 4112us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.242007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.242033 (+ 26us) server_negotiation.cc:407] Connection header received
0809 13:14:12.242133 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.242140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.242303 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.242475 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:12.243272 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.244665 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.247440 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.248677 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.248860 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.257536 (+ 8676us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.257624 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.257781 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3948,"thread_start_us":3780,"threads_started":1}
I20260809 13:14:12.261734 7502 tablet_service.cc:1467] Processing CreateTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e (DEFAULT_TABLE table=test-table [id=f602dac8400d4cb7b160d27b799d789d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:12.263164 7502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21bd50b337f14d34a9b47b8455c31e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:12.263912 8037 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Bootstrap starting.
I20260809 13:14:12.266162 8038 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Bootstrap starting.
I20260809 13:14:12.268090 8037 tablet_bootstrap.cc:654] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:12.269563 8038 tablet_bootstrap.cc:654] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:12.269613 8037 log.cc:824] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:12.270671 8038 log.cc:824] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:12.272539 8037 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: No bootstrap required, opened a new log
I20260809 13:14:12.272845 8037 ts_tablet_manager.cc:1397] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 13:14:12.273586 8038 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: No bootstrap required, opened a new log
I20260809 13:14:12.273845 8038 ts_tablet_manager.cc:1397] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 13:14:12.276885 8041 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Bootstrap starting.
I20260809 13:14:12.279618 8041 tablet_bootstrap.cc:654] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:12.280514 8041 log.cc:824] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:12.282795 8041 tablet_bootstrap.cc:492] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: No bootstrap required, opened a new log
I20260809 13:14:12.283021 8041 ts_tablet_manager.cc:1397] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 13:14:12.283556 8037 raft_consensus.cc:348] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.284052 8037 raft_consensus.cc:374] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:12.284170 8037 raft_consensus.cc:729] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e83cb8c59b4065beb39cf6ef1e706f, State: Initialized, Role: FOLLOWER
I20260809 13:14:12.284090 8038 raft_consensus.cc:348] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.284520 8038 raft_consensus.cc:374] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:12.284626 8038 raft_consensus.cc:729] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef59e39e4c94a89803b1f159edf09b4, State: Initialized, Role: FOLLOWER
I20260809 13:14:12.284933 8037 consensus_queue.cc:260] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.285211 8038 consensus_queue.cc:260] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.286119 8009 heartbeater.cc:503] Master 127.10.181.190:40135 was elected leader, sending a full tablet report...
I20260809 13:14:12.286875 8037 ts_tablet_manager.cc:1428] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260809 13:14:12.287751 7810 heartbeater.cc:503] Master 127.10.181.190:40135 was elected leader, sending a full tablet report...
I20260809 13:14:12.288444 8038 ts_tablet_manager.cc:1428] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 13:14:12.293421 8041 raft_consensus.cc:348] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.293797 8041 raft_consensus.cc:374] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:12.293875 8041 raft_consensus.cc:729] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae354f35f5e48748370544e73e24098, State: Initialized, Role: FOLLOWER
I20260809 13:14:12.294296 8041 consensus_queue.cc:260] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.295778 8041 ts_tablet_manager.cc:1428] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260809 13:14:12.360728 7610 tablet.cc:2369] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:12.382943 7811 tablet.cc:2369] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:12.405669 8011 tablet.cc:2369] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:12.461017 8049 raft_consensus.cc:482] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:12.461330 8049 raft_consensus.cc:504] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.463060 8049 leader_election.cc:288] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ef59e39e4c94a89803b1f159edf09b4 (127.10.181.130:33519), a5e83cb8c59b4065beb39cf6ef1e706f (127.10.181.131:43407)
I20260809 13:14:12.469849 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.463512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:44457 (local address 127.10.181.130:33519)
0809 13:14:12.463739 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.463745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.464451 (+ 706us) server_negotiation.cc:407] Connection header received
0809 13:14:12.464646 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.464651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.464711 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.464783 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:12.465463 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.466188 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.468006 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.468810 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.468954 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.469026 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.469462 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.469595 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:14:12.469866 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.463644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:35009 (local address 127.10.181.131:43407)
0809 13:14:12.463952 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.463957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.464556 (+ 599us) server_negotiation.cc:407] Connection header received
0809 13:14:12.464731 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.464735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.464795 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.464871 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:12.465531 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.466280 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.468082 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.468917 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.469069 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.469142 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.469514 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.469649 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260809 13:14:12.469980 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.463332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:33519 (local address 127.10.181.129:44457)
0809 13:14:12.464311 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.464331 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.464484 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.464810 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.464814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.464838 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:12.465294 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.465304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.466309 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.466315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.467839 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.467852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.469036 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.469043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.469284 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.469311 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.469386 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.469515 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":34,"thread_start_us":584,"threads_started":1}
I20260809 13:14:12.470002 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.463503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:43407 (local address 127.10.181.129:35009)
0809 13:14:12.464449 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.464467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:12.464602 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:12.464903 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:12.464908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:12.464945 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:12.465375 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.465385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.466420 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.466426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.467917 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:12.467930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.469211 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.469217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:12.469346 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.469371 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 13:14:12.469485 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 13:14:12.469609 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":1,"thread_start_us":644,"threads_started":1}
I20260809 13:14:12.470842 7942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bd50b337f14d34a9b47b8455c31e0e" candidate_uuid: "0ae354f35f5e48748370544e73e24098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" is_pre_election: true
I20260809 13:14:12.471127 7748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bd50b337f14d34a9b47b8455c31e0e" candidate_uuid: "0ae354f35f5e48748370544e73e24098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef59e39e4c94a89803b1f159edf09b4" is_pre_election: true
I20260809 13:14:12.471297 7942 raft_consensus.cc:2454] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae354f35f5e48748370544e73e24098 in term 0.
I20260809 13:14:12.471551 7748 raft_consensus.cc:2454] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae354f35f5e48748370544e73e24098 in term 0.
I20260809 13:14:12.471920 7431 leader_election.cc:302] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae354f35f5e48748370544e73e24098, a5e83cb8c59b4065beb39cf6ef1e706f; no voters:
I20260809 13:14:12.472337 8049 raft_consensus.cc:2781] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:12.472465 8049 raft_consensus.cc:482] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:12.472563 8049 raft_consensus.cc:3037] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:12.474756 8049 raft_consensus.cc:504] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.475288 8049 leader_election.cc:288] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [CANDIDATE]: Term 1 election: Requested vote from peers 9ef59e39e4c94a89803b1f159edf09b4 (127.10.181.130:33519), a5e83cb8c59b4065beb39cf6ef1e706f (127.10.181.131:43407)
I20260809 13:14:12.475682 7748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bd50b337f14d34a9b47b8455c31e0e" candidate_uuid: "0ae354f35f5e48748370544e73e24098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef59e39e4c94a89803b1f159edf09b4"
I20260809 13:14:12.475694 7942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21bd50b337f14d34a9b47b8455c31e0e" candidate_uuid: "0ae354f35f5e48748370544e73e24098" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f"
I20260809 13:14:12.475845 7748 raft_consensus.cc:3037] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:12.475855 7942 raft_consensus.cc:3037] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:12.478018 7748 raft_consensus.cc:2454] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae354f35f5e48748370544e73e24098 in term 1.
I20260809 13:14:12.478099 7942 raft_consensus.cc:2454] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae354f35f5e48748370544e73e24098 in term 1.
I20260809 13:14:12.478500 7432 leader_election.cc:302] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae354f35f5e48748370544e73e24098, 9ef59e39e4c94a89803b1f159edf09b4; no voters:
I20260809 13:14:12.478798 8049 raft_consensus.cc:2781] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:12.479254 8049 raft_consensus.cc:686] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 1 LEADER]: Becoming Leader. State: Replica: 0ae354f35f5e48748370544e73e24098, State: Running, Role: LEADER
I20260809 13:14:12.479717 8049 consensus_queue.cc:237] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } }
I20260809 13:14:12.484228 7322 catalog_manager.cc:5495] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae354f35f5e48748370544e73e24098 (127.10.181.129). New cstate: current_term: 1 leader_uuid: "0ae354f35f5e48748370544e73e24098" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae354f35f5e48748370544e73e24098" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 45865 } health_report { overall_health: HEALTHY } } }
I20260809 13:14:12.547672 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.542442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.10.181.129:45865)
0809 13:14:12.542648 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.542653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.542672 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:14:12.542813 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.542817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.542898 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.543017 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:12.543333 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.544050 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.545410 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.545850 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.545966 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.546579 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:12.547083 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:12.547250 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.547409 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.547457 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:14:12.555567 7942 raft_consensus.cc:1261] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 1 FOLLOWER]: Refusing update from remote peer 0ae354f35f5e48748370544e73e24098: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:12.555666 7748 raft_consensus.cc:1261] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 1 FOLLOWER]: Refusing update from remote peer 0ae354f35f5e48748370544e73e24098: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:12.556450 8052 consensus_queue.cc:1035] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e83cb8c59b4065beb39cf6ef1e706f" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 43407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:12.556814 8049 consensus_queue.cc:1035] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef59e39e4c94a89803b1f159edf09b4" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 33519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:12.572456 8058 mvcc.cc:205] Tried to move back new op lower bound from 7316608010458226688 to 7316608010160201728. Current Snapshot: MvccSnapshot[applied={T|T < 7316608010458226688}]
I20260809 13:14:12.582279 7321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:12.582522 7321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:12.586174 7321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f802cab36f114951930c18c28d07a9d0: Sending DeleteTablet for 3 replicas of tablet 21bd50b337f14d34a9b47b8455c31e0e
I20260809 13:14:12.587170 7911 tablet_service.cc:1514] Processing DeleteTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:12 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:57440
I20260809 13:14:12.587306 7712 tablet_service.cc:1514] Processing DeleteTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:12 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:33284
I20260809 13:14:12.588178 7502 tablet_service.cc:1514] Processing DeleteTablet for tablet 21bd50b337f14d34a9b47b8455c31e0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:12 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:46140
I20260809 13:14:12.590864 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 7410
I20260809 13:14:12.606914 8073 tablet_replica.cc:330] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: stopping tablet replica
I20260809 13:14:12.607357 8073 raft_consensus.cc:2229] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:12.607887 8073 raft_consensus.cc:2258] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:12.609510 8074 tablet_replica.cc:330] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: stopping tablet replica
I20260809 13:14:12.609661 8071 tablet_replica.cc:330] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: stopping tablet replica
I20260809 13:14:12.609855 8074 raft_consensus.cc:2229] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:12.610038 8071 raft_consensus.cc:2229] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:12.610428 8074 raft_consensus.cc:2258] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:12.610492 8071 raft_consensus.cc:2258] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:12.612428 8071 ts_tablet_manager.cc:1905] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:12.615877 8073 ts_tablet_manager.cc:1905] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 13:14:12 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:14:12.619781 8071 ts_tablet_manager.cc:1918] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:12.619932 8071 log.cc:1196] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/21bd50b337f14d34a9b47b8455c31e0e
I20260809 13:14:12.620743 8071 ts_tablet_manager.cc:1939] T 21bd50b337f14d34a9b47b8455c31e0e P a5e83cb8c59b4065beb39cf6ef1e706f: Deleting consensus metadata
I20260809 13:14:12.622855 7302 catalog_manager.cc:4841] TS a5e83cb8c59b4065beb39cf6ef1e706f (127.10.181.131:43407): tablet 21bd50b337f14d34a9b47b8455c31e0e (table test-table [id=f602dac8400d4cb7b160d27b799d789d]) successfully deleted
I20260809 13:14:12.623172 8074 ts_tablet_manager.cc:1905] T 21bd50b337f14d34a9b47b8455c31e0e P 0ae354f35f5e48748370544e73e24098: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:12.627045 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.592450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.10.181.129:45865)
0809 13:14:12.592661 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.592668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.603963 (+ 11295us) server_negotiation.cc:407] Connection header received
0809 13:14:12.604372 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.604378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.604484 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.604560 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:12.610784 (+ 6224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.611806 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.613160 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.613862 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.614003 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.619366 (+ 5363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:12.619395 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:12.619412 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:12.619454 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:12.623525 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.624378 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.624391 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.624395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.624580 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.625039 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.625044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.625047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.625543 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:12.625838 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.626435 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.626671 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:14:12.627872 8073 ts_tablet_manager.cc:1918] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:12.628098 8073 log.cc:1196] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/21bd50b337f14d34a9b47b8455c31e0e
I20260809 13:14:12.628459 7601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:12.628744 8073 ts_tablet_manager.cc:1939] T 21bd50b337f14d34a9b47b8455c31e0e P 9ef59e39e4c94a89803b1f159edf09b4: Deleting consensus metadata
I20260809 13:14:12.631064 7303 catalog_manager.cc:4841] TS 9ef59e39e4c94a89803b1f159edf09b4 (127.10.181.130:33519): tablet 21bd50b337f14d34a9b47b8455c31e0e (table test-table [id=f602dac8400d4cb7b160d27b799d789d]) successfully deleted
I20260809 13:14:12.726023 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 7410
W20260809 13:14:12.753536 7303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:45865 (error 108)
I20260809 13:14:12.754326 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 7615
I20260809 13:14:12.755522 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.754822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46160)
0809 13:14:12.755264 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.755366 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":206,"threads_started":1}
W20260809 13:14:12.755720 7303 catalog_manager.cc:4584] TS 0ae354f35f5e48748370544e73e24098 (127.10.181.129:45865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21bd50b337f14d34a9b47b8455c31e0e: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Aug 09 13:14:12 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:14:12.768782 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.755061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.10.181.130:33519)
0809 13:14:12.755279 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.755285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.755569 (+ 284us) server_negotiation.cc:407] Connection header received
0809 13:14:12.755796 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.755802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.755903 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.756012 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:12.757335 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.758127 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.759356 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.759717 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.759802 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.763666 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:12.763680 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:12.763695 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:12.763734 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:12.766432 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.767036 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.767042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.767044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.767174 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.767464 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.767468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.767470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.767858 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:12.768096 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.768353 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.768530 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 13:14:12.769400 7804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:12.809401 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.808965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46164)
0809 13:14:12.809161 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.809261 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 13:14:12.810461 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.810079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46170)
0809 13:14:12.810252 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.810326 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 13:14:12.856673 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 7615
I20260809 13:14:12.870718 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.870294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46176)
0809 13:14:12.870484 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.870569 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 13:14:12.871681 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.871326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46190)
0809 13:14:12.871470 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.871541 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 13:14:12.881119 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 7819
Aug 09 13:14:12 dist-test-slave-42z9 krb5kdc[7257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:14:12.895545 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.881873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.10.181.131:43407)
0809 13:14:12.882061 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:12.882067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:12.882088 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:14:12.882242 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:12.882247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:12.882330 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:12.882413 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:12.883765 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.884473 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.885839 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:12.886234 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:12.886344 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:12.890538 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:12.890554 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:12.890577 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:12.890612 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:12.893158 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.893851 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.893859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.893861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.893997 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:12.894315 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:12.894318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:12.894320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:12.894650 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:12.894869 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:12.895077 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 13:14:12.895233 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:14:12.896136 8003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:12.952203 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.951773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46206)
0809 13:14:12.951969 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.952054 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 13:14:12.953245 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:12.952834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:45865 (local address 127.0.0.1:46220)
0809 13:14:12.953042 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:12.953114 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 13:14:12.983315 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 7819
I20260809 13:14:13.006851 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 7277
I20260809 13:14:13.007912 7359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:13.117070 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 7277
2026-08-09T13:14:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4645 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T13:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:13Z Disabled control of system clock
I20260809 13:14:13.213302 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39539
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:13.539007 8091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:13.539274 8091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:13.539361 8091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:13.546469 8091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:13.546562 8091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:13.546612 8091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:13.546667 8091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:13.546715 8091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:13.555672 8091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:13.556828 8091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:13.558552 8091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:13.569583 8098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:13.569763 8103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:13.569660 8100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:13.570283 8091 server_base.cc:1034] running on GCE node
I20260809 13:14:13.570968 8091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:13.572219 8091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:13.573418 8091 hybrid_clock.cc:648] HybridClock initialized: now 1786281253573360 us; error 54 us; skew 500 ppm
I20260809 13:14:13.573766 8091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:13.575846 8091 webserver.cc:460] Webserver started at http://127.10.181.190:42269/ using document root <none> and password file <none>
I20260809 13:14:13.576486 8091 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:13.576586 8091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:13.576884 8091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:13.579651 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "eb4bf62bf98b4d5fabb76d62a15afe60"
format_stamp: "Formatted at 2026-08-09 13:14:13 on dist-test-slave-42z9"
server_key: "26e6e52b66d37e38a9c475489869abab"
server_key_iv: "db24f9041b13bbcce47ad6690d7759c4"
server_key_version: "encryptionkey@0"
I20260809 13:14:13.580403 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "eb4bf62bf98b4d5fabb76d62a15afe60"
format_stamp: "Formatted at 2026-08-09 13:14:13 on dist-test-slave-42z9"
server_key: "26e6e52b66d37e38a9c475489869abab"
server_key_iv: "db24f9041b13bbcce47ad6690d7759c4"
server_key_version: "encryptionkey@0"
I20260809 13:14:13.585112 8091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:14:13.588328 8109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:13.589557 8091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 13:14:13.589717 8091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "eb4bf62bf98b4d5fabb76d62a15afe60"
format_stamp: "Formatted at 2026-08-09 13:14:13 on dist-test-slave-42z9"
server_key: "26e6e52b66d37e38a9c475489869abab"
server_key_iv: "db24f9041b13bbcce47ad6690d7759c4"
server_key_version: "encryptionkey@0"
I20260809 13:14:13.589915 8091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:13.628840 8091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:13.629683 8091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:13.629940 8091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:13.646798 8091 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39539
I20260809 13:14:13.646822 8185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39539 every 8 connection(s)
I20260809 13:14:13.648036 8091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:13.651575 8186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:13.656682 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 8091
I20260809 13:14:13.656857 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:13.657392 10966 external_mini_cluster.cc:1428] Setting key 0ccccf014cf9541283ee5f62b2438181
I20260809 13:14:13.662457 8186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: Bootstrap starting.
I20260809 13:14:13.666054 8186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:13.667186 8186 log.cc:824] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:13.670135 8186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: No bootstrap required, opened a new log
I20260809 13:14:13.670662 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:13.659292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.10.181.190:39539)
0809 13:14:13.660294 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:13.660303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:13.660354 (+ 51us) server_negotiation.cc:407] Connection header received
0809 13:14:13.662013 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:13.662036 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:13.662447 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:13.662761 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:13.663347 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:13.664615 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:13.666498 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:13.667143 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:13.667273 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:13.668525 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:13.668532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:13.668545 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:13.669023 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:13.669113 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:13.669139 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:13.669228 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:13.669479 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 13:14:13.669777 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":353,"threads_started":1}
I20260809 13:14:13.677248 8186 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } }
I20260809 13:14:13.677595 8186 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:13.677690 8186 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4bf62bf98b4d5fabb76d62a15afe60, State: Initialized, Role: FOLLOWER
I20260809 13:14:13.678231 8186 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } }
I20260809 13:14:13.678452 8186 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:13.678579 8186 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:13.678728 8186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:13.680770 8186 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } }
I20260809 13:14:13.681267 8186 leader_election.cc:302] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb4bf62bf98b4d5fabb76d62a15afe60; no voters:
I20260809 13:14:13.681874 8186 leader_election.cc:288] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:13.682052 8193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:13.683082 8193 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [term 1 LEADER]: Becoming Leader. State: Replica: eb4bf62bf98b4d5fabb76d62a15afe60, State: Running, Role: LEADER
I20260809 13:14:13.683537 8193 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } }
I20260809 13:14:13.683871 8186 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:13.686161 8195 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } } }
I20260809 13:14:13.686590 8195 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:13.687053 8197 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb4bf62bf98b4d5fabb76d62a15afe60. Latest consensus state: current_term: 1 leader_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4bf62bf98b4d5fabb76d62a15afe60" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39539 } } }
I20260809 13:14:13.687340 8197 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:13.688165 8206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:13.694057 8206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:13.702162 8206 catalog_manager.cc:1347] Generated new cluster ID: 373bba7e611541b0b1a9a2a3e8fb5e30
I20260809 13:14:13.702271 8206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:13.736975 8206 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:13.738413 8206 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:13.749724 8206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: Generated new TSK 0
I20260809 13:14:13.750514 8206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:13.762723 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--builtin_ntp_servers=127.10.181.148:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:14.114154 8216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:14.114418 8216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:14.114545 8216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:14.121802 8216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:14.121899 8216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:14.122056 8216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:14.131634 8216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:14.132903 8216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:14.134827 8216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:14.147646 8223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:14.147735 8225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:14.147684 8227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:14.148548 8216 server_base.cc:1034] running on GCE node
I20260809 13:14:14.149115 8216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:14.150018 8216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:14.151239 8216 hybrid_clock.cc:648] HybridClock initialized: now 1786281254151196 us; error 35 us; skew 500 ppm
I20260809 13:14:14.151535 8216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:14.153678 8216 webserver.cc:460] Webserver started at http://127.10.181.129:40653/ using document root <none> and password file <none>
I20260809 13:14:14.154330 8216 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:14.154426 8216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:14.154734 8216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:14.157512 8216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "bab47f3683804551b263235ac847d76f"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "45cee4474c22e2409e3779baca08dc3f"
server_key_iv: "357d8aa2a52bb000e4a7121bbab02134"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.158286 8216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "bab47f3683804551b263235ac847d76f"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "45cee4474c22e2409e3779baca08dc3f"
server_key_iv: "357d8aa2a52bb000e4a7121bbab02134"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.163206 8216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:14:14.166546 8239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.167752 8216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 13:14:14.167899 8216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "bab47f3683804551b263235ac847d76f"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "45cee4474c22e2409e3779baca08dc3f"
server_key_iv: "357d8aa2a52bb000e4a7121bbab02134"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.168095 8216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:14.183218 8216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:14.183929 8216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:14.184197 8216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:14.185118 8216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:14.186556 8216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:14.186643 8216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.186722 8216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:14.186781 8216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.217064 8216 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:46195
I20260809 13:14:14.217104 8411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:46195 every 8 connection(s)
I20260809 13:14:14.218322 8216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:14.219385 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 8216
I20260809 13:14:14.219568 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:14.220098 10966 external_mini_cluster.cc:1428] Setting key 6fe4ce6d6608c86ab41d5390e022f615
I20260809 13:14:14.223407 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--builtin_ntp_servers=127.10.181.148:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:14.230834 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:14.221019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:52139 (local address 127.10.181.190:39539)
0809 13:14:14.221469 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:14.221475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:14.222829 (+ 1354us) server_negotiation.cc:407] Connection header received
0809 13:14:14.224618 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:14.224624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:14.224699 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:14.224792 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:14.225874 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.226616 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.228146 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.228567 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.228681 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:14.229999 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:14.230005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:14.230009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:14.230027 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:14.230070 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:14.230082 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:14.230167 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:14.230376 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 13:14:14.230476 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":202,"threads_started":1}
I20260809 13:14:14.231740 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:14.221654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39539 (local address 127.10.181.129:52139)
0809 13:14:14.222682 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:14.222732 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:14.224365 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:14.225026 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:14.225035 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:14.225117 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:14.225713 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:14.225732 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.226783 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.226787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:14.227995 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:14.228009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.228801 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.228810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:14.229011 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:14.229706 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:14.229709 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:14.229747 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:14.229820 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:14.230170 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:14.230183 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:14.230191 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:14:14.230335 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 13:14:14.230724 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":292,"threads_started":1}
I20260809 13:14:14.233238 8412 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39539
I20260809 13:14:14.233546 8412 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:14.234292 8412 heartbeater.cc:511] Master 127.10.181.190:39539 requested a full tablet report, sending...
I20260809 13:14:14.236486 8138 ts_manager.cc:194] Registered new tserver with Master: bab47f3683804551b263235ac847d76f (127.10.181.129:46195)
I20260809 13:14:14.238094 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.129:52139
W20260809 13:14:14.555684 8418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:14.555934 8418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:14.556025 8418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:14.562950 8418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:14.563038 8418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:14.563144 8418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:14.571758 8418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:14.572947 8418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:14.574676 8418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:14.587450 8428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:14.587426 8427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:14.587920 8430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:14.588326 8418 server_base.cc:1034] running on GCE node
I20260809 13:14:14.588876 8418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:14.589807 8418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:14.591003 8418 hybrid_clock.cc:648] HybridClock initialized: now 1786281254590944 us; error 51 us; skew 500 ppm
I20260809 13:14:14.591288 8418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:14.593432 8418 webserver.cc:460] Webserver started at http://127.10.181.130:42037/ using document root <none> and password file <none>
I20260809 13:14:14.594136 8418 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:14.594235 8418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:14.594524 8418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:14.597229 8418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "81b7fffe3bf14c649e1e57416e4a94a6"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "fbff14672683e434f6f7a8dbce0ba2db"
server_key_iv: "fe2bc48082959e2b8a759e26d140853a"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.598023 8418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "81b7fffe3bf14c649e1e57416e4a94a6"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "fbff14672683e434f6f7a8dbce0ba2db"
server_key_iv: "fe2bc48082959e2b8a759e26d140853a"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.602815 8418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:14:14.606014 8442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.607192 8418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 13:14:14.607339 8418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "81b7fffe3bf14c649e1e57416e4a94a6"
format_stamp: "Formatted at 2026-08-09 13:14:14 on dist-test-slave-42z9"
server_key: "fbff14672683e434f6f7a8dbce0ba2db"
server_key_iv: "fe2bc48082959e2b8a759e26d140853a"
server_key_version: "encryptionkey@0"
I20260809 13:14:14.607520 8418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:14.632747 8418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:14.633520 8418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:14.633807 8418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:14.634724 8418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:14.636076 8418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:14.636170 8418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.636291 8418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:14.636359 8418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:14.666987 8418 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:38253
I20260809 13:14:14.667048 8596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:38253 every 8 connection(s)
I20260809 13:14:14.668157 8418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:14.675257 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 8418
I20260809 13:14:14.675437 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:14.675959 10966 external_mini_cluster.cc:1428] Setting key d1d53e4d0ca9ce1edcdd82f1e42188f1
I20260809 13:14:14.678934 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--builtin_ntp_servers=127.10.181.148:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:14.680537 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:14.670697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:51915 (local address 127.10.181.190:39539)
0809 13:14:14.670859 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:14.670863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:14.672118 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 13:14:14.673921 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:14.673925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:14.673991 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:14.674074 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:14.675084 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.676134 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.677933 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.678482 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.678624 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:14.679831 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:14.679836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:14.679841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:14.679863 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:14.679902 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:14.679917 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:14.679997 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:14.680168 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 13:14:14.680257 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:14:14.681496 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:14.671085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39539 (local address 127.10.181.130:51915)
0809 13:14:14.671983 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:14.672018 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:14.673693 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:14.674275 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:14.674283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:14.674372 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:14.674922 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:14.674939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.676315 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.676320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:14.677788 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:14.677806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:14.678674 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:14.678679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:14.678868 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:14.679537 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:14.679539 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:14.679583 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:14.679657 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:14.680003 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:14.680009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:14.680014 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 13:14:14.680148 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 13:14:14.680420 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":270,"threads_started":1}
I20260809 13:14:14.682824 8598 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39539
I20260809 13:14:14.683116 8598 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:14.683972 8598 heartbeater.cc:511] Master 127.10.181.190:39539 requested a full tablet report, sending...
I20260809 13:14:14.685478 8138 ts_manager.cc:194] Registered new tserver with Master: 81b7fffe3bf14c649e1e57416e4a94a6 (127.10.181.130:38253)
I20260809 13:14:14.686424 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.130:51915
W20260809 13:14:15.016745 8605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:15.016997 8605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:15.017118 8605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:15.024084 8605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:15.024181 8605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:15.024328 8605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:15.033061 8605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:15.034337 8605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:15.036113 8605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:15.048753 8614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:15.048815 8616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:15.048918 8612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:15.050110 8605 server_base.cc:1034] running on GCE node
I20260809 13:14:15.050690 8605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:15.051596 8605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:15.052810 8605 hybrid_clock.cc:648] HybridClock initialized: now 1786281255052764 us; error 35 us; skew 500 ppm
I20260809 13:14:15.053122 8605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:15.055253 8605 webserver.cc:460] Webserver started at http://127.10.181.131:45433/ using document root <none> and password file <none>
I20260809 13:14:15.055922 8605 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:15.056020 8605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:15.056321 8605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:15.059142 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "f31f31d193be468787199ff652939f82"
format_stamp: "Formatted at 2026-08-09 13:14:15 on dist-test-slave-42z9"
server_key: "8cc367b11bed7830d6b35d28c9ab9ce5"
server_key_iv: "c3b063df6780830bb4a70a4c88a865ba"
server_key_version: "encryptionkey@0"
I20260809 13:14:15.059912 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "f31f31d193be468787199ff652939f82"
format_stamp: "Formatted at 2026-08-09 13:14:15 on dist-test-slave-42z9"
server_key: "8cc367b11bed7830d6b35d28c9ab9ce5"
server_key_iv: "c3b063df6780830bb4a70a4c88a865ba"
server_key_version: "encryptionkey@0"
I20260809 13:14:15.064682 8605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 13:14:15.068034 8625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:15.069283 8605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:14:15.069434 8605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "f31f31d193be468787199ff652939f82"
format_stamp: "Formatted at 2026-08-09 13:14:15 on dist-test-slave-42z9"
server_key: "8cc367b11bed7830d6b35d28c9ab9ce5"
server_key_iv: "c3b063df6780830bb4a70a4c88a865ba"
server_key_version: "encryptionkey@0"
I20260809 13:14:15.069650 8605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:15.086334 8605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:15.087029 8605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:15.087273 8605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:15.088130 8605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:15.089627 8605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:15.089715 8605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:15.089793 8605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:15.089854 8605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:15.119536 8605 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:35503
I20260809 13:14:15.119586 8782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:35503 every 8 connection(s)
I20260809 13:14:15.120817 8605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:15.130119 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 8605
I20260809 13:14:15.130295 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:15.130793 10966 external_mini_cluster.cc:1428] Setting key a6e94d9b31c7521afc997702e381b6cf
I20260809 13:14:15.132861 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.123395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:54539 (local address 127.10.181.190:39539)
0809 13:14:15.123579 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.123585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.124971 (+ 1386us) server_negotiation.cc:407] Connection header received
0809 13:14:15.126745 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.126750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.126829 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.126916 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:15.127944 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.128691 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.130275 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.130722 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.130836 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.132068 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:15.132081 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:15.132084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:15.132101 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:15.132136 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:15.132147 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:15.132221 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.132429 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.132532 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:14:15.133824 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.123939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39539 (local address 127.10.181.131:54539)
0809 13:14:15.124820 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.124862 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.126477 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.127147 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.127155 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.127236 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:15.127780 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.127796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.128837 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.128840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.130131 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.130144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.130941 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.130949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.131158 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.131769 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:15.131771 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:15.131809 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:15.131880 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:15.132236 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:15.132247 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:15.132255 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.132403 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.132779 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260809 13:14:15.135082 8783 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39539
I20260809 13:14:15.135342 8783 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:15.136040 8783 heartbeater.cc:511] Master 127.10.181.190:39539 requested a full tablet report, sending...
I20260809 13:14:15.137337 8138 ts_manager.cc:194] Registered new tserver with Master: f31f31d193be468787199ff652939f82 (127.10.181.131:35503)
I20260809 13:14:15.138199 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.131:54539
I20260809 13:14:15.145576 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:15.157205 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.149568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.10.181.190:39539)
0809 13:14:15.149738 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.149743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.150188 (+ 445us) server_negotiation.cc:407] Connection header received
0809 13:14:15.150341 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.150346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.150432 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.150525 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:15.151117 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.152123 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.154281 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.154890 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.155019 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.156296 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:15.156302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:15.156308 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:15.156329 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:15.156365 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:15.156381 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:15.156527 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.156905 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.156977 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:14:15.160763 8138 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:15.167470 8138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:15.195169 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.179895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.10.181.131:35503)
0809 13:14:15.180692 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.180701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.185967 (+ 5266us) server_negotiation.cc:407] Connection header received
0809 13:14:15.186278 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.186285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.186503 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.186661 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.187612 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.189225 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.192800 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194010 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.194197 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.194418 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.194665 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.194827 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":414,"threads_started":1}
I20260809 13:14:15.195822 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.179452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:35503 (local address 127.0.0.1:56952)
0809 13:14:15.185864 (+ 6412us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.185880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.186064 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.186964 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.186968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.186992 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.187394 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.187408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.189504 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.189509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.192526 (+ 3017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.192545 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194192 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.194199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.194436 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.194489 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.194622 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.194818 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6279,"mutex_wait_us":696,"thread_start_us":5247,"threads_started":1}
I20260809 13:14:15.195911 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.179392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.10.181.129:46195)
0809 13:14:15.185686 (+ 6294us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.185695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.185726 (+ 31us) server_negotiation.cc:407] Connection header received
0809 13:14:15.185825 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.185833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.186092 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.188314 (+ 2222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.188526 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.189929 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.192644 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194105 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.194639 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.194832 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.195462 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.195615 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6114,"thread_start_us":5986,"threads_started":1}
I20260809 13:14:15.195878 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.179203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56814)
0809 13:14:15.185054 (+ 5851us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.185103 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.185279 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.186327 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.186334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.186364 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.186897 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.186911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.190089 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.190095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.192350 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.192366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194322 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.194329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.194507 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.194944 (+ 437us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.195041 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.195165 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5619,"mutex_wait_us":386,"thread_start_us":6402,"threads_started":1}
I20260809 13:14:15.199230 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.185998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:38253 (local address 127.0.0.1:46144)
0809 13:14:15.191243 (+ 5245us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.191260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.191383 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.192082 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.192087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.192126 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.192663 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.192676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194472 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.194476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.197017 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.197030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.198697 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.198702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.198819 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.198841 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.198901 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.198979 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5118,"mutex_wait_us":4,"thread_start_us":4972,"threads_started":1}
I20260809 13:14:15.199425 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.186254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.10.181.130:38253)
0809 13:14:15.190917 (+ 4663us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.190935 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.191512 (+ 577us) server_negotiation.cc:407] Connection header received
0809 13:14:15.191620 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.191627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.191821 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.191985 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.192849 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.194322 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.197190 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.198521 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.198689 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.198882 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.198943 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.199110 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4501,"thread_start_us":4380,"threads_started":1}
I20260809 13:14:15.200268 8312 tablet_service.cc:1467] Processing CreateTablet for tablet ffb581a6f3934381b95183692e82c1f3 (DEFAULT_TABLE table=test-table [id=d0f6d56ec917495497ea598b2931a271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:15.201741 8312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb581a6f3934381b95183692e82c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:15.202514 8510 tablet_service.cc:1467] Processing CreateTablet for tablet ffb581a6f3934381b95183692e82c1f3 (DEFAULT_TABLE table=test-table [id=d0f6d56ec917495497ea598b2931a271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:15.203588 8510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb581a6f3934381b95183692e82c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:15.208950 8696 tablet_service.cc:1467] Processing CreateTablet for tablet ffb581a6f3934381b95183692e82c1f3 (DEFAULT_TABLE table=test-table [id=d0f6d56ec917495497ea598b2931a271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:15.210440 8696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb581a6f3934381b95183692e82c1f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:15.212700 8811 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Bootstrap starting.
I20260809 13:14:15.216418 8811 tablet_bootstrap.cc:654] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:15.216692 8814 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Bootstrap starting.
I20260809 13:14:15.218024 8811 log.cc:824] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:15.220175 8814 tablet_bootstrap.cc:654] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:15.220980 8811 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: No bootstrap required, opened a new log
I20260809 13:14:15.221328 8811 ts_tablet_manager.cc:1397] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 13:14:15.221618 8814 log.cc:824] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:15.230494 8817 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Bootstrap starting.
I20260809 13:14:15.232008 8814 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: No bootstrap required, opened a new log
I20260809 13:14:15.232277 8814 ts_tablet_manager.cc:1397] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260809 13:14:15.231981 8811 raft_consensus.cc:348] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.232438 8811 raft_consensus.cc:374] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:15.232553 8811 raft_consensus.cc:729] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab47f3683804551b263235ac847d76f, State: Initialized, Role: FOLLOWER
I20260809 13:14:15.233213 8811 consensus_queue.cc:260] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.234400 8412 heartbeater.cc:503] Master 127.10.181.190:39539 was elected leader, sending a full tablet report...
I20260809 13:14:15.235040 8811 ts_tablet_manager.cc:1428] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260809 13:14:15.235675 8817 tablet_bootstrap.cc:654] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:15.236745 8817 log.cc:824] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:15.239705 8817 tablet_bootstrap.cc:492] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: No bootstrap required, opened a new log
I20260809 13:14:15.239974 8817 ts_tablet_manager.cc:1397] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 13:14:15.241994 8814 raft_consensus.cc:348] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.242339 8814 raft_consensus.cc:374] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:15.242434 8814 raft_consensus.cc:729] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b7fffe3bf14c649e1e57416e4a94a6, State: Initialized, Role: FOLLOWER
I20260809 13:14:15.243000 8814 consensus_queue.cc:260] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.244057 8598 heartbeater.cc:503] Master 127.10.181.190:39539 was elected leader, sending a full tablet report...
I20260809 13:14:15.244517 8814 ts_tablet_manager.cc:1428] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 13:14:15.248507 8817 raft_consensus.cc:348] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.248868 8817 raft_consensus.cc:374] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:15.248947 8817 raft_consensus.cc:729] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31f31d193be468787199ff652939f82, State: Initialized, Role: FOLLOWER
I20260809 13:14:15.249498 8817 consensus_queue.cc:260] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.250437 8783 heartbeater.cc:503] Master 127.10.181.190:39539 was elected leader, sending a full tablet report...
I20260809 13:14:15.250823 8817 ts_tablet_manager.cc:1428] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
I20260809 13:14:15.266707 8819 raft_consensus.cc:482] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:15.266999 8819 raft_consensus.cc:504] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.268651 8819 leader_election.cc:288] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f31f31d193be468787199ff652939f82 (127.10.181.131:35503), 81b7fffe3bf14c649e1e57416e4a94a6 (127.10.181.130:38253)
I20260809 13:14:15.274317 8822 raft_consensus.cc:482] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:15.274641 8822 raft_consensus.cc:504] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.275938 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.269121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53217 (local address 127.10.181.130:38253)
0809 13:14:15.269333 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.269340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.270091 (+ 751us) server_negotiation.cc:407] Connection header received
0809 13:14:15.270269 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.270273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.270360 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.270450 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.271102 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.271846 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.273965 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.274884 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.275020 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.275099 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.275518 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.275665 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:14:15.276052 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.269140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:54125 (local address 127.10.181.131:35503)
0809 13:14:15.269372 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.269379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.270021 (+ 642us) server_negotiation.cc:407] Connection header received
0809 13:14:15.270184 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.270190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.270262 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.270327 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.270920 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.271722 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.273973 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.275198 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.275360 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.275423 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.275703 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.275826 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 13:14:15.276108 8824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.268862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:35503 (local address 127.10.181.129:54125)
0809 13:14:15.269901 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.269922 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.270048 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.270365 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.270370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.270393 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.270786 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.270796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.271855 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.271861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.273763 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.273776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.275383 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.275389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.275551 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.275581 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.275658 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.275777 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":137,"thread_start_us":667,"threads_started":1}
I20260809 13:14:15.276639 8823 raft_consensus.cc:482] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:15.276755 8822 leader_election.cc:288] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab47f3683804551b263235ac847d76f (127.10.181.129:46195), f31f31d193be468787199ff652939f82 (127.10.181.131:35503)
I20260809 13:14:15.277146 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.269180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:38253 (local address 127.10.181.129:53217)
0809 13:14:15.269962 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.269978 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.270114 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.270478 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.270482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.270515 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.270936 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.270949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.271990 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.271996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.273746 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.273759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.275130 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.275138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.275295 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.275328 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.276582 (+ 1254us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.276709 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":21,"thread_start_us":485,"threads_started":1}
I20260809 13:14:15.276996 8823 raft_consensus.cc:504] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.277398 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "bab47f3683804551b263235ac847d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31f31d193be468787199ff652939f82" is_pre_election: true
I20260809 13:14:15.278036 8724 raft_consensus.cc:2454] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab47f3683804551b263235ac847d76f in term 0.
I20260809 13:14:15.278795 8241 leader_election.cc:302] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bab47f3683804551b263235ac847d76f, f31f31d193be468787199ff652939f82; no voters:
I20260809 13:14:15.278934 8823 leader_election.cc:288] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab47f3683804551b263235ac847d76f (127.10.181.129:46195), 81b7fffe3bf14c649e1e57416e4a94a6 (127.10.181.130:38253)
I20260809 13:14:15.279243 8819 raft_consensus.cc:2781] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:15.279407 8819 raft_consensus.cc:482] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:15.279506 8819 raft_consensus.cc:3037] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:15.282370 8819 raft_consensus.cc:504] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.284440 8819 leader_election.cc:288] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [CANDIDATE]: Term 1 election: Requested vote from peers f31f31d193be468787199ff652939f82 (127.10.181.131:35503), 81b7fffe3bf14c649e1e57416e4a94a6 (127.10.181.130:38253)
I20260809 13:14:15.287235 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.277662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:33673 (local address 127.10.181.131:35503)
0809 13:14:15.277909 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.277915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.279217 (+ 1302us) server_negotiation.cc:407] Connection header received
0809 13:14:15.279462 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.279468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.279542 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.279642 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.280405 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.281480 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.284390 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.285747 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.285890 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.285960 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.286791 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.286935 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 13:14:15.288303 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.277186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:35503 (local address 127.10.181.130:33673)
0809 13:14:15.279045 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.279132 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.279276 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.279790 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.279795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.279827 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.280248 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.280259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.281642 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.281647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.284110 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.284125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.286390 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.286396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.286602 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.286637 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.287827 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.287979 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1478,"threads_started":1}
I20260809 13:14:15.290292 8541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "bab47f3683804551b263235ac847d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" is_pre_election: true
I20260809 13:14:15.290759 8541 raft_consensus.cc:2454] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab47f3683804551b263235ac847d76f in term 0.
I20260809 13:14:15.291510 8541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "bab47f3683804551b263235ac847d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b7fffe3bf14c649e1e57416e4a94a6"
I20260809 13:14:15.291738 8541 raft_consensus.cc:3037] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:15.294304 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "bab47f3683804551b263235ac847d76f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31f31d193be468787199ff652939f82"
I20260809 13:14:15.294508 8724 raft_consensus.cc:3037] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:15.295362 8541 raft_consensus.cc:2454] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab47f3683804551b263235ac847d76f in term 1.
I20260809 13:14:15.295830 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.278673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:60807 (local address 127.10.181.129:46195)
0809 13:14:15.278842 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.278848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.288785 (+ 9937us) server_negotiation.cc:407] Connection header received
0809 13:14:15.289024 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.289030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.289125 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.289184 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.289743 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.290846 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.293530 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.294687 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.294805 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.294868 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.295439 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.295563 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 13:14:15.296393 8246 leader_election.cc:302] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b7fffe3bf14c649e1e57416e4a94a6, bab47f3683804551b263235ac847d76f; no voters:
I20260809 13:14:15.296746 8819 raft_consensus.cc:2781] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:15.297669 8724 raft_consensus.cc:2454] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab47f3683804551b263235ac847d76f in term 1.
I20260809 13:14:15.300623 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.278145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.10.181.130:60807)
0809 13:14:15.288695 (+ 10550us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.288715 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.288879 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.289360 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.289365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.289392 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.289586 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.289595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.291115 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.291121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.293297 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.293319 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.295062 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.295068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.295255 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.295287 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.300165 (+ 4878us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.300311 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10402,"thread_start_us":575,"threads_started":1}
I20260809 13:14:15.300976 8724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31f31d193be468787199ff652939f82" is_pre_election: true
I20260809 13:14:15.301312 8724 raft_consensus.cc:2379] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81b7fffe3bf14c649e1e57416e4a94a6 in current term 1: Already voted for candidate bab47f3683804551b263235ac847d76f in this term.
I20260809 13:14:15.301666 8342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab47f3683804551b263235ac847d76f" is_pre_election: true
I20260809 13:14:15.303006 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.283477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.10.181.131:39819)
0809 13:14:15.283640 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.283657 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.283744 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.296531 (+ 12787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.296537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.296565 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.296759 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.296771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.298343 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.298347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.300027 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.300038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.302351 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.302355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.302496 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.302518 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.302601 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.302729 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 13:14:15.303253 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.283630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:39819 (local address 127.10.181.129:46195)
0809 13:14:15.296028 (+ 12398us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.296033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.296049 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:14:15.296101 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.296106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.296169 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.296247 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.297043 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.298133 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.300176 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.302094 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.302239 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.302282 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.302645 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.302758 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12304,"thread_start_us":15262,"threads_started":1}
I20260809 13:14:15.303831 8341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "f31f31d193be468787199ff652939f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab47f3683804551b263235ac847d76f" is_pre_election: true
I20260809 13:14:15.305567 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.284331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:41457 (local address 127.10.181.130:38253)
0809 13:14:15.286110 (+ 1779us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.286116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.298365 (+ 12249us) server_negotiation.cc:407] Connection header received
0809 13:14:15.298618 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.298624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.298689 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.298747 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:15.299543 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.300715 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.303278 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.304517 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.304658 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.304727 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.305248 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.305337 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1656}
I20260809 13:14:15.306041 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.283778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:38253 (local address 127.10.181.131:41457)
0809 13:14:15.298277 (+ 14499us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.298295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:15.298461 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:15.298929 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:15.298933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:15.298957 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:15.299347 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.299359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.300883 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.300887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.303047 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:15.303065 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.304884 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.304891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:15.305076 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.305109 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 13:14:15.305193 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 13:14:15.305327 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9968,"thread_start_us":10079,"threads_started":1}
I20260809 13:14:15.306877 8541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb581a6f3934381b95183692e82c1f3" candidate_uuid: "f31f31d193be468787199ff652939f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" is_pre_election: true
I20260809 13:14:15.307144 8541 raft_consensus.cc:2379] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f31f31d193be468787199ff652939f82 in current term 1: Already voted for candidate bab47f3683804551b263235ac847d76f in this term.
I20260809 13:14:15.307624 8630 leader_election.cc:302] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f31f31d193be468787199ff652939f82; no voters: 81b7fffe3bf14c649e1e57416e4a94a6, bab47f3683804551b263235ac847d76f
I20260809 13:14:15.308019 8823 raft_consensus.cc:2726] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 13:14:15.308487 8819 raft_consensus.cc:686] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 1 LEADER]: Becoming Leader. State: Replica: bab47f3683804551b263235ac847d76f, State: Running, Role: LEADER
I20260809 13:14:15.308950 8819 consensus_queue.cc:237] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } }
I20260809 13:14:15.312911 8446 leader_election.cc:302] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81b7fffe3bf14c649e1e57416e4a94a6; no voters: bab47f3683804551b263235ac847d76f, f31f31d193be468787199ff652939f82
I20260809 13:14:15.313325 8822 raft_consensus.cc:2726] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 13:14:15.314033 8137 catalog_manager.cc:5495] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f reported cstate change: term changed from 0 to 1, leader changed from <none> to bab47f3683804551b263235ac847d76f (127.10.181.129). New cstate: current_term: 1 leader_uuid: "bab47f3683804551b263235ac847d76f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab47f3683804551b263235ac847d76f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 } health_report { overall_health: UNKNOWN } } }
I20260809 13:14:15.348829 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.343185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.10.181.129:46195)
0809 13:14:15.343368 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.343374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.343461 (+ 87us) server_negotiation.cc:407] Connection header received
0809 13:14:15.343686 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.343690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.343782 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.343929 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:15.344320 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.345073 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.346606 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.347016 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.347127 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.347753 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:15.348256 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:15.348416 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.348560 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.348602 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 13:14:15.356608 8541 raft_consensus.cc:1261] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Refusing update from remote peer bab47f3683804551b263235ac847d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:15.356649 8724 raft_consensus.cc:1261] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Refusing update from remote peer bab47f3683804551b263235ac847d76f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:15.357406 8834 consensus_queue.cc:1035] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "81b7fffe3bf14c649e1e57416e4a94a6" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:15.357803 8819 consensus_queue.cc:1035] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31f31d193be468787199ff652939f82" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:14:15.372834 8785 tablet.cc:2369] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:15.373315 8841 mvcc.cc:205] Tried to move back new op lower bound from 7316608021931769856 to 7316608021749276672. Current Snapshot: MvccSnapshot[applied={T|T < 7316608021931769856}]
I20260809 13:14:15.389950 8138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:15.390208 8138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:15.393668 8138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb4bf62bf98b4d5fabb76d62a15afe60: Sending DeleteTablet for 3 replicas of tablet ffb581a6f3934381b95183692e82c1f3
I20260809 13:14:15.394917 8510 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb581a6f3934381b95183692e82c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:15 UTC) from {username='slave'} at 127.0.0.1:46144
I20260809 13:14:15.395140 8696 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb581a6f3934381b95183692e82c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:15 UTC) from {username='slave'} at 127.0.0.1:56952
I20260809 13:14:15.395841 8312 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb581a6f3934381b95183692e82c1f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:15 UTC) from {username='slave'} at 127.0.0.1:56814
I20260809 13:14:15.396147 8859 tablet_replica.cc:330] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: stopping tablet replica
I20260809 13:14:15.396634 8859 raft_consensus.cc:2229] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:15.397254 8859 raft_consensus.cc:2258] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:15.397550 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 8216
I20260809 13:14:15.399590 8859 ts_tablet_manager.cc:1905] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:15.407258 8859 ts_tablet_manager.cc:1918] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:15.407552 8859 log.cc:1196] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/ffb581a6f3934381b95183692e82c1f3
I20260809 13:14:15.408186 8859 ts_tablet_manager.cc:1939] T ffb581a6f3934381b95183692e82c1f3 P f31f31d193be468787199ff652939f82: Deleting consensus metadata
I20260809 13:14:15.410346 8857 tablet_replica.cc:330] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: stopping tablet replica
I20260809 13:14:15.410354 8111 catalog_manager.cc:4841] TS f31f31d193be468787199ff652939f82 (127.10.181.131:35503): tablet ffb581a6f3934381b95183692e82c1f3 (table test-table [id=d0f6d56ec917495497ea598b2931a271]) successfully deleted
I20260809 13:14:15.410820 8857 raft_consensus.cc:2229] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:15.411269 8857 raft_consensus.cc:2258] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:15.413069 8857 ts_tablet_manager.cc:1905] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:15.417702 8857 ts_tablet_manager.cc:1918] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:15.417870 8857 log.cc:1196] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/ffb581a6f3934381b95183692e82c1f3
I20260809 13:14:15.418430 8857 ts_tablet_manager.cc:1939] T ffb581a6f3934381b95183692e82c1f3 P 81b7fffe3bf14c649e1e57416e4a94a6: Deleting consensus metadata
I20260809 13:14:15.419616 8860 tablet_replica.cc:330] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: stopping tablet replica
I20260809 13:14:15.420301 8117 catalog_manager.cc:4841] TS 81b7fffe3bf14c649e1e57416e4a94a6 (127.10.181.130:38253): tablet ffb581a6f3934381b95183692e82c1f3 (table test-table [id=d0f6d56ec917495497ea598b2931a271]) successfully deleted
I20260809 13:14:15.420727 8860 raft_consensus.cc:2229] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:15.421487 8860 raft_consensus.cc:2258] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:15.423520 8860 ts_tablet_manager.cc:1905] T ffb581a6f3934381b95183692e82c1f3 P bab47f3683804551b263235ac847d76f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:15.425969 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.398823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.10.181.129:46195)
0809 13:14:15.399038 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.399043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.418192 (+ 19149us) server_negotiation.cc:407] Connection header received
0809 13:14:15.418436 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.418442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.418552 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.418669 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:15.419305 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.420424 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.422273 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.422963 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.423092 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.424247 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:15.424258 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:15.424275 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:15.425113 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:15.425176 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:15.425201 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:15.425327 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.425501 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.425618 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:14:15.426693 8404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:15.523885 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 8216
W20260809 13:14:15.549016 8117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:46195 (error 108)
I20260809 13:14:15.549242 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 8418
I20260809 13:14:15.550979 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.550545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56828)
0809 13:14:15.550723 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.550838 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 13:14:15.551296 8117 catalog_manager.cc:4584] TS bab47f3683804551b263235ac847d76f (127.10.181.129:46195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ffb581a6f3934381b95183692e82c1f3: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
I20260809 13:14:15.555929 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.550110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.10.181.130:38253)
0809 13:14:15.550307 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.550314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.550337 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:14:15.550471 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.550475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.550578 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.550672 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:15.551120 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.551871 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.553126 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.553539 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.553663 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.554687 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:15.554696 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:15.554710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:15.555244 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:15.555291 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:15.555310 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:15.555418 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.555542 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.555650 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 13:14:15.556560 8592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:15.604848 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.604333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56840)
0809 13:14:15.604554 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.604661 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 13:14:15.605822 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.605460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56856)
0809 13:14:15.605584 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.605673 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 13:14:15.644601 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 8418
I20260809 13:14:15.666061 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.665629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56870)
0809 13:14:15.665815 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.665899 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:14:15.667091 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.666750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56876)
0809 13:14:15.666898 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.666973 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 13:14:15.675112 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 8605
I20260809 13:14:15.681715 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.675795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56958 (local address 127.10.181.131:35503)
0809 13:14:15.675974 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:15.675980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:15.676027 (+ 47us) server_negotiation.cc:407] Connection header received
0809 13:14:15.676227 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:15.676230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:15.676328 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:15.676413 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:15.676846 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.677595 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.678870 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:15.679268 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:15.679402 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:15.680354 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:15.680365 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:15.680381 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:15.680976 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:15.681025 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:15.681050 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:15.681162 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:15.681292 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 13:14:15.681393 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 13:14:15.682349 8777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:15.748486 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.747995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56884)
0809 13:14:15.748226 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.748327 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 13:14:15.749639 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:15.749198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46195 (local address 127.0.0.1:56888)
0809 13:14:15.749401 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:15.749520 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 13:14:15.770617 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 8605
I20260809 13:14:15.795626 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 8091
I20260809 13:14:15.796687 8179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:15.903605 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 8091
2026-08-09T13:14:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2783 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:16 dist-test-slave-42z9 krb5kdc[8871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:16 dist-test-slave-42z9 krb5kdc[8871](info): set up 2 sockets
Aug 09 13:14:16 dist-test-slave-42z9 krb5kdc[8871](info): commencing operation
krb5kdc: starting...
W20260809 13:14:17.128291 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:17 dist-test-slave-42z9 krb5kdc[8871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:17.414245 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40131
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:17.740820 8890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:17.741104 8890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:17.741245 8890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:17.748238 8890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:17.748346 8890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:17.748435 8890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:17.748509 8890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:17.748570 8890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:17.758124 8890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:40131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:17.759351 8890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:17.760613 8890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:17.771780 8902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:17.771795 8903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:17.771919 8905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:17.772732 8890 server_base.cc:1034] running on GCE node
I20260809 13:14:17.773419 8890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:17.774771 8890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:17.775982 8890 hybrid_clock.cc:648] HybridClock initialized: now 1786281257775916 us; error 49 us; skew 500 ppm
I20260809 13:14:17.776315 8890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:17 dist-test-slave-42z9 krb5kdc[8871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:17.779270 8890 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:17.780627 8890 webserver.cc:460] Webserver started at http://127.10.181.190:37629/ using document root <none> and password file <none>
I20260809 13:14:17.781275 8890 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:17.781390 8890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:17.781684 8890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:17.784560 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "f163c5eacca54adcaca7e292813be5ca"
format_stamp: "Formatted at 2026-08-09 13:14:17 on dist-test-slave-42z9"
server_key: "b61492a01c579fe6b6cd9c3875ed0278"
server_key_iv: "8c586773226bf6d6afdcbd7cf935ce69"
server_key_version: "encryptionkey@0"
I20260809 13:14:17.785276 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "f163c5eacca54adcaca7e292813be5ca"
format_stamp: "Formatted at 2026-08-09 13:14:17 on dist-test-slave-42z9"
server_key: "b61492a01c579fe6b6cd9c3875ed0278"
server_key_iv: "8c586773226bf6d6afdcbd7cf935ce69"
server_key_version: "encryptionkey@0"
I20260809 13:14:17.790199 8890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 13:14:17.793526 8912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:17.794751 8890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:14:17.794924 8890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "f163c5eacca54adcaca7e292813be5ca"
format_stamp: "Formatted at 2026-08-09 13:14:17 on dist-test-slave-42z9"
server_key: "b61492a01c579fe6b6cd9c3875ed0278"
server_key_iv: "8c586773226bf6d6afdcbd7cf935ce69"
server_key_version: "encryptionkey@0"
I20260809 13:14:17.795101 8890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:17.820181 8890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:17.824189 8890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:17.824441 8890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:17.841140 8890 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:40131
I20260809 13:14:17.841163 8983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:40131 every 8 connection(s)
I20260809 13:14:17.842478 8890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:17.846094 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:17.847877 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 8890
I20260809 13:14:17.848042 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:17.848465 10966 external_mini_cluster.cc:1428] Setting key 9c3eb88a367db5cc9ce7b6125fc72852
I20260809 13:14:17.859676 8985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: Bootstrap starting.
Aug 09 13:14:17 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:17.863291 8985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:17.864578 8985 log.cc:824] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:17.867868 8985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: No bootstrap required, opened a new log
I20260809 13:14:17.871824 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:17.850179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.10.181.190:40131)
0809 13:14:17.851032 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:17.851043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:17.851087 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:14:17.852648 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:17.852674 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:17.853025 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:17.853376 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:17.855199 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:17.856480 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:17.858314 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:17.858975 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:17.859098 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:17.863689 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:17.863701 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:17.863719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:17.863774 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:17.867477 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:17.868182 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:17.868190 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:17.868197 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:17.868431 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:17.868780 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:17.868785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:17.868788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:17.869492 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:17.869798 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:17.870105 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 13:14:17.870544 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":384,"threads_started":1}
I20260809 13:14:17.876438 8985 raft_consensus.cc:348] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } }
I20260809 13:14:17.876765 8985 raft_consensus.cc:374] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:17.876842 8985 raft_consensus.cc:729] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f163c5eacca54adcaca7e292813be5ca, State: Initialized, Role: FOLLOWER
I20260809 13:14:17.877300 8985 consensus_queue.cc:260] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } }
I20260809 13:14:17.877561 8985 raft_consensus.cc:388] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:17.877672 8985 raft_consensus.cc:482] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:17.877816 8985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:17.880090 8985 raft_consensus.cc:504] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } }
I20260809 13:14:17.880519 8985 leader_election.cc:302] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f163c5eacca54adcaca7e292813be5ca; no voters:
I20260809 13:14:17.881057 8985 leader_election.cc:288] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:17.881173 8995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:17.881821 8995 raft_consensus.cc:686] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [term 1 LEADER]: Becoming Leader. State: Replica: f163c5eacca54adcaca7e292813be5ca, State: Running, Role: LEADER
I20260809 13:14:17.882368 8995 consensus_queue.cc:237] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } }
I20260809 13:14:17.882928 8985 sys_catalog.cc:564] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:17.885017 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f163c5eacca54adcaca7e292813be5ca. Latest consensus state: current_term: 1 leader_uuid: "f163c5eacca54adcaca7e292813be5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } } }
I20260809 13:14:17.885411 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:17.886736 8996 sys_catalog.cc:455] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f163c5eacca54adcaca7e292813be5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f163c5eacca54adcaca7e292813be5ca" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40131 } } }
I20260809 13:14:17.887101 8996 sys_catalog.cc:458] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:17.888320 9012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:17.892990 9012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:17.901690 9012 catalog_manager.cc:1347] Generated new cluster ID: a2fa0444d8c642b89506280a338282cd
I20260809 13:14:17.901791 9012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:17.923321 9012 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:17.924521 9012 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:17.931993 9012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: Generated new TSK 0
I20260809 13:14:17.932703 9012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.181.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
Entry for principal kudu/127.10.181.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab.
I20260809 13:14:18.008630 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:18.344280 9032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:18.344552 9032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:18.344702 9032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:18.351627 9032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:18.351734 9032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:18.351877 9032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:18.361119 9032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.129.keytab
--principal=kudu/127.10.181.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:18.362310 9032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:18.363544 9032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:18.375895 9046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:18.375931 9043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:18.376192 9042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:18.377162 9032 server_base.cc:1034] running on GCE node
I20260809 13:14:18.377770 9032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:18.378615 9032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:18.379810 9032 hybrid_clock.cc:648] HybridClock initialized: now 1786281258379753 us; error 49 us; skew 500 ppm
I20260809 13:14:18.380098 9032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:18 dist-test-slave-42z9 krb5kdc[8871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281258, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:18.383109 9032 init.cc:377] Logged in from keytab as kudu/127.10.181.129@KRBTEST.COM (short username kudu)
I20260809 13:14:18.384402 9032 webserver.cc:460] Webserver started at http://127.10.181.129:34321/ using document root <none> and password file <none>
I20260809 13:14:18.385015 9032 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:18.385113 9032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:18.385360 9032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:18.388250 9032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "19c0fd5a10cc4b938cff0282df66bb0b"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "c4009cefd65783060aaca6c94dceed0b"
server_key_iv: "5ea067321e34e2abee6a95fb9764406a"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.388952 9032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "19c0fd5a10cc4b938cff0282df66bb0b"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "c4009cefd65783060aaca6c94dceed0b"
server_key_iv: "5ea067321e34e2abee6a95fb9764406a"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.393692 9032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260809 13:14:18.396827 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.398046 9032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 13:14:18.398206 9032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "19c0fd5a10cc4b938cff0282df66bb0b"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "c4009cefd65783060aaca6c94dceed0b"
server_key_iv: "5ea067321e34e2abee6a95fb9764406a"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.398370 9032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:18.418643 9032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:18.422210 9032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:18.422461 9032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:18.423362 9032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:18.424604 9032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:18.424681 9032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.424798 9032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:18.424846 9032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.455052 9032 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:41425
I20260809 13:14:18.455119 9230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:41425 every 8 connection(s)
I20260809 13:14:18.456305 9032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:18.457329 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 9032
I20260809 13:14:18.457542 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:18.458074 10966 external_mini_cluster.cc:1428] Setting key ee2ab6c5fc7da92c20868ce367e4c721
Aug 09 13:14:18 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281258, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.129@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:18.477967 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:18.459117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:53307 (local address 127.10.181.190:40131)
0809 13:14:18.459606 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:18.459612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:18.460720 (+ 1108us) server_negotiation.cc:407] Connection header received
0809 13:14:18.463144 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:18.463148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:18.463213 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:18.463295 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:18.465145 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.465899 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.467924 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.468333 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.468428 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:18.472371 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:18.472376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:18.472379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:18.472412 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:18.474826 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:18.475654 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:18.475659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:18.475661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:18.475787 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:18.476221 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:18.476224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:18.476227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:18.476493 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:18.476639 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:18.477335 (+ 696us) server_negotiation.cc:299] Negotiation successful
0809 13:14:18.477598 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
I20260809 13:14:18.478879 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:18.459694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40131 (local address 127.10.181.129:53307)
0809 13:14:18.460531 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:18.460589 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:18.462852 (+ 2263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:18.463547 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:18.463557 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:18.464229 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:18.464990 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:18.465014 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.466041 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.466046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:18.467794 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:18.467809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.468557 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.468564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:18.468827 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:18.469513 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:18.469547 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:18.472148 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:18.474962 (+ 2814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:18.474970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:18.474986 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:18.475534 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:18.475897 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:18.475902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:18.475906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:18.476097 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:18.476620 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:18.476628 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:18.476955 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 13:14:18.477203 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 13:14:18.477734 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":309,"threads_started":1}
I20260809 13:14:18.480446 9233 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40131
I20260809 13:14:18.480741 9233 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:18.481460 9233 heartbeater.cc:511] Master 127.10.181.190:40131 requested a full tablet report, sending...
I20260809 13:14:18.483779 8934 ts_manager.cc:194] Registered new tserver with Master: 19c0fd5a10cc4b938cff0282df66bb0b (127.10.181.129:41425)
I20260809 13:14:18.485692 8934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.129@KRBTEST.COM'} at 127.10.181.129:53307
WARNING: no policy specified for kudu/127.10.181.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
Entry for principal kudu/127.10.181.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab.
I20260809 13:14:18.526173 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:18.854538 9239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:18.854780 9239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:18.854885 9239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:18.861915 9239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:18.862010 9239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:18.862136 9239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:18.871352 9239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.130.keytab
--principal=kudu/127.10.181.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:18.872462 9239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:18.873708 9239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:18.886296 9246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:18.886212 9249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:18.886260 9247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:18.887008 9239 server_base.cc:1034] running on GCE node
I20260809 13:14:18.887552 9239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:18.888363 9239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:18.889515 9239 hybrid_clock.cc:648] HybridClock initialized: now 1786281258889460 us; error 48 us; skew 500 ppm
I20260809 13:14:18.889794 9239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:18 dist-test-slave-42z9 krb5kdc[8871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281258, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:18.892670 9239 init.cc:377] Logged in from keytab as kudu/127.10.181.130@KRBTEST.COM (short username kudu)
I20260809 13:14:18.894042 9239 webserver.cc:460] Webserver started at http://127.10.181.130:38049/ using document root <none> and password file <none>
I20260809 13:14:18.894764 9239 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:18.894882 9239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:18.895171 9239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:18.897903 9239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "a75c9265a15a41b9b3f3949ba0a28115"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "decbe84e7bc01a654fb5789b8671c4a7"
server_key_iv: "76735154f361e13c2ae4ff003683a955"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.898676 9239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "a75c9265a15a41b9b3f3949ba0a28115"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "decbe84e7bc01a654fb5789b8671c4a7"
server_key_iv: "76735154f361e13c2ae4ff003683a955"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.903653 9239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:14:18.906992 9256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.908202 9239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 13:14:18.908351 9239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "a75c9265a15a41b9b3f3949ba0a28115"
format_stamp: "Formatted at 2026-08-09 13:14:18 on dist-test-slave-42z9"
server_key: "decbe84e7bc01a654fb5789b8671c4a7"
server_key_iv: "76735154f361e13c2ae4ff003683a955"
server_key_version: "encryptionkey@0"
I20260809 13:14:18.908529 9239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:18.925477 9239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:18.929046 9239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:18.929296 9239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:18.930226 9239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:18.931499 9239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:18.931574 9239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.931707 9239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:18.931757 9239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:18.960228 9239 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:45259
I20260809 13:14:18.960268 9422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:45259 every 8 connection(s)
I20260809 13:14:18.961477 9239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:18.962389 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 9239
I20260809 13:14:18.962580 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:18.963124 10966 external_mini_cluster.cc:1428] Setting key f4e1c26451ea304f659f52b1ac5bee8d
Aug 09 13:14:18 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281258, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.130@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:18.982398 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:18.964321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:56935 (local address 127.10.181.190:40131)
0809 13:14:18.964603 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:18.964609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:18.966060 (+ 1451us) server_negotiation.cc:407] Connection header received
0809 13:14:18.968490 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:18.968494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:18.968562 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:18.968658 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:18.970207 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.970919 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.972547 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.972986 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.973097 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:18.976383 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:18.976388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:18.976393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:18.976430 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:18.979288 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:18.980090 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:18.980094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:18.980097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:18.980245 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:18.980656 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:18.980660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:18.980664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:18.980971 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:18.981105 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:18.981844 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 13:14:18.982073 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 13:14:18.983359 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:18.964903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40131 (local address 127.10.181.130:56935)
0809 13:14:18.965887 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:18.965949 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:18.968187 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:18.968920 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:18.968929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:18.969423 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:18.970026 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:18.970045 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.971147 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.971152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:18.972397 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:18.972410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:18.973227 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:18.973236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:18.973463 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:18.974072 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:18.974099 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:18.976159 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:18.979436 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:18.979445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:18.979459 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:18.979958 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:18.980338 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:18.980342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:18.980345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:18.980548 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:18.981112 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:18.981120 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:18.981420 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 13:14:18.981708 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 13:14:18.982205 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":378,"threads_started":1}
I20260809 13:14:18.984823 9424 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40131
I20260809 13:14:18.985105 9424 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:18.985836 9424 heartbeater.cc:511] Master 127.10.181.190:40131 requested a full tablet report, sending...
I20260809 13:14:18.987344 8934 ts_manager.cc:194] Registered new tserver with Master: a75c9265a15a41b9b3f3949ba0a28115 (127.10.181.130:45259)
I20260809 13:14:18.988133 8934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.130@KRBTEST.COM'} at 127.10.181.130:56935
WARNING: no policy specified for kudu/127.10.181.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
Entry for principal kudu/127.10.181.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab.
I20260809 13:14:19.032087 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:19.366539 9430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:19.366788 9430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:19.366919 9430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:19.373960 9430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:19.374058 9430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:19.374204 9430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:19.383472 9430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.131.keytab
--principal=kudu/127.10.181.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:40131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:19.384641 9430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:19.385914 9430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:19.398229 9439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:19.398259 9443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:19.398478 9438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:19.399387 9430 server_base.cc:1034] running on GCE node
I20260809 13:14:19.399966 9430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:19.400797 9430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:19.401983 9430 hybrid_clock.cc:648] HybridClock initialized: now 1786281259401939 us; error 34 us; skew 500 ppm
I20260809 13:14:19.402303 9430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:19 dist-test-slave-42z9 krb5kdc[8871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:19.405213 9430 init.cc:377] Logged in from keytab as kudu/127.10.181.131@KRBTEST.COM (short username kudu)
I20260809 13:14:19.406606 9430 webserver.cc:460] Webserver started at http://127.10.181.131:39263/ using document root <none> and password file <none>
I20260809 13:14:19.407196 9430 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:19.407294 9430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:19.407560 9430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:19.410372 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "e4068d80ac73421488bbd90709a14536"
format_stamp: "Formatted at 2026-08-09 13:14:19 on dist-test-slave-42z9"
server_key: "75faa0b94f2258e85fcb249796f4665d"
server_key_iv: "0bb427f1f3ed9554ae53fe9e25160b3a"
server_key_version: "encryptionkey@0"
I20260809 13:14:19.411059 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "e4068d80ac73421488bbd90709a14536"
format_stamp: "Formatted at 2026-08-09 13:14:19 on dist-test-slave-42z9"
server_key: "75faa0b94f2258e85fcb249796f4665d"
server_key_iv: "0bb427f1f3ed9554ae53fe9e25160b3a"
server_key_version: "encryptionkey@0"
I20260809 13:14:19.416002 9430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 13:14:19.419322 9455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:19.420552 9430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:14:19.420694 9430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "e4068d80ac73421488bbd90709a14536"
format_stamp: "Formatted at 2026-08-09 13:14:19 on dist-test-slave-42z9"
server_key: "75faa0b94f2258e85fcb249796f4665d"
server_key_iv: "0bb427f1f3ed9554ae53fe9e25160b3a"
server_key_version: "encryptionkey@0"
I20260809 13:14:19.420869 9430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:19.447633 9430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:19.451663 9430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:19.451916 9430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:19.452828 9430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:19.454115 9430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:19.454186 9430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:19.454322 9430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:19.454373 9430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:19.484886 9430 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:38729
I20260809 13:14:19.484932 9621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:38729 every 8 connection(s)
I20260809 13:14:19.486435 9430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:19.488641 9233 heartbeater.cc:503] Master 127.10.181.190:40131 was elected leader, sending a full tablet report...
I20260809 13:14:19.489728 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 9430
I20260809 13:14:19.489912 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:19.490465 10966 external_mini_cluster.cc:1428] Setting key 5fd08a93650872c275e10ebdbcde4c77
Aug 09 13:14:19 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.131@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:19.505347 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.489066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:49817 (local address 127.10.181.190:40131)
0809 13:14:19.490636 (+ 1570us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.490642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.490663 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:14:19.492598 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.492602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.492661 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.492744 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:19.494263 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.495033 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.496465 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.496845 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.496942 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.500010 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:19.500014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:19.500018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:19.500048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:19.502470 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.503094 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.503097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.503100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.503217 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.503671 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.503676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.503679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.503971 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:19.504130 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.504820 (+ 690us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.505032 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1450,"thread_start_us":2000,"threads_started":1}
I20260809 13:14:19.506157 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.489478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:40131 (local address 127.10.181.131:49817)
0809 13:14:19.490276 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.490330 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.492346 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.492988 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.492999 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.493562 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:19.494105 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.494122 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.495165 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.495169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.496356 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.496372 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.497013 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.497018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.497195 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.497826 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:14:19.497848 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:19.499841 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:19.502599 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:19.502606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:19.502616 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:19.503001 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:19.503289 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:14:19.503292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:14:19.503295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:14:19.503542 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:14:19.504093 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:19.504101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:19.504405 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.504692 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.505112 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":303,"threads_started":1}
I20260809 13:14:19.507308 9622 heartbeater.cc:347] Connected to a master server at 127.10.181.190:40131
I20260809 13:14:19.507557 9622 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:19.508184 9622 heartbeater.cc:511] Master 127.10.181.190:40131 requested a full tablet report, sending...
I20260809 13:14:19.509418 8934 ts_manager.cc:194] Registered new tserver with Master: e4068d80ac73421488bbd90709a14536 (127.10.181.131:38729)
I20260809 13:14:19.510222 8934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.181.131@KRBTEST.COM'} at 127.10.181.131:49817
I20260809 13:14:19.517009 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:19.532076 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.520864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.10.181.190:40131)
0809 13:14:19.521016 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.521020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.521514 (+ 494us) server_negotiation.cc:407] Connection header received
0809 13:14:19.521665 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.521669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.521741 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.521816 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:19.523115 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.523866 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.525134 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.525551 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.525647 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.527818 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:19.527822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:19.527825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:19.527854 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:19.530069 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.530642 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.530645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.530648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.530751 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.531013 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.531015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.531017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.531289 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:19.531462 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.531714 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.531875 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:14:19.535408 8934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:19.542109 8934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:19.564119 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.555060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.10.181.130:45259)
0809 13:14:19.556511 (+ 1451us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.556519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.557256 (+ 737us) server_negotiation.cc:407] Connection header received
0809 13:14:19.557379 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.557386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.557639 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.557824 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:19.558603 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559778 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.562107 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.563031 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.563183 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.563360 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.563644 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.563804 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":1167,"threads_started":1}
I20260809 13:14:19.564194 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.554544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:45259 (local address 127.0.0.1:42884)
0809 13:14:19.556982 (+ 2438us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.557000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.557125 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.558058 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.558062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.558087 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:19.558419 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.558432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559928 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.559932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.561952 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.561970 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.563230 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.563236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.563417 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.563453 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.563569 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.563748 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2299,"mutex_wait_us":63,"thread_start_us":2134,"threads_started":1}
I20260809 13:14:19.564841 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.553398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41425 (local address 127.0.0.1:43442)
0809 13:14:19.554157 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.554197 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.554382 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.556776 (+ 2394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.556784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.556820 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:19.557307 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.557320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559165 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.559171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.561984 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.562000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.563795 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.563802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.563996 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.564031 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.564098 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.564436 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":166,"thread_start_us":353,"threads_started":1}
I20260809 13:14:19.565025 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.554506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.10.181.131:38729)
0809 13:14:19.556245 (+ 1739us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.556254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.556286 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:14:19.556381 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.556387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.556633 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.556831 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:19.557663 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559051 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.562176 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.563557 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.563753 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.563918 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.564553 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.564733 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1563,"thread_start_us":1410,"threads_started":1}
I20260809 13:14:19.565569 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.553995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:38729 (local address 127.0.0.1:34032)
0809 13:14:19.555174 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.555194 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.555310 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.556879 (+ 1569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.556888 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.556914 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:19.557426 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.557464 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559242 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.559247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.561987 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.562000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.564062 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.564068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.564253 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.564281 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.564352 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.564454 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":187,"thread_start_us":734,"threads_started":1}
I20260809 13:14:19.566306 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.553810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.10.181.129:41425)
0809 13:14:19.555882 (+ 2072us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.555892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.555921 (+ 29us) server_negotiation.cc:407] Connection header received
0809 13:14:19.556018 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.556025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.556516 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.557534 (+ 1018us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:19.557773 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.559036 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.562176 (+ 3140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.563558 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.565598 (+ 2040us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.565764 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.565843 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.565998 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1893,"thread_start_us":1738,"threads_started":1}
I20260809 13:14:19.568229 9331 tablet_service.cc:1467] Processing CreateTablet for tablet 75955e306fd043be96ae89762b9f47ac (DEFAULT_TABLE table=test-table [id=d67bf274008d4746a014dcbe311fe834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:19.569814 9331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75955e306fd043be96ae89762b9f47ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:19.569293 9534 tablet_service.cc:1467] Processing CreateTablet for tablet 75955e306fd043be96ae89762b9f47ac (DEFAULT_TABLE table=test-table [id=d67bf274008d4746a014dcbe311fe834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:19.570802 9534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75955e306fd043be96ae89762b9f47ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:19.571828 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 75955e306fd043be96ae89762b9f47ac (DEFAULT_TABLE table=test-table [id=d67bf274008d4746a014dcbe311fe834]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:19.573192 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75955e306fd043be96ae89762b9f47ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:19.584903 9645 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Bootstrap starting.
I20260809 13:14:19.585605 9644 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Bootstrap starting.
I20260809 13:14:19.586372 9646 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Bootstrap starting.
I20260809 13:14:19.589071 9644 tablet_bootstrap.cc:654] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:19.589174 9646 tablet_bootstrap.cc:654] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:19.590185 9646 log.cc:824] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:19.590315 9644 log.cc:824] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:19.592738 9645 tablet_bootstrap.cc:654] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:19.593359 9646 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: No bootstrap required, opened a new log
I20260809 13:14:19.593627 9644 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: No bootstrap required, opened a new log
I20260809 13:14:19.593670 9646 ts_tablet_manager.cc:1397] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260809 13:14:19.593873 9644 ts_tablet_manager.cc:1397] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 13:14:19.594041 9645 log.cc:824] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:19.596922 9645 tablet_bootstrap.cc:492] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: No bootstrap required, opened a new log
I20260809 13:14:19.597186 9645 ts_tablet_manager.cc:1397] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260809 13:14:19.601256 9644 raft_consensus.cc:348] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.601644 9644 raft_consensus.cc:374] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:19.601744 9644 raft_consensus.cc:729] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c0fd5a10cc4b938cff0282df66bb0b, State: Initialized, Role: FOLLOWER
I20260809 13:14:19.602314 9644 consensus_queue.cc:260] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.603672 9646 raft_consensus.cc:348] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.604126 9646 raft_consensus.cc:374] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:19.604259 9646 raft_consensus.cc:729] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4068d80ac73421488bbd90709a14536, State: Initialized, Role: FOLLOWER
I20260809 13:14:19.604930 9646 consensus_queue.cc:260] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.605147 9644 ts_tablet_manager.cc:1428] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260809 13:14:19.606163 9622 heartbeater.cc:503] Master 127.10.181.190:40131 was elected leader, sending a full tablet report...
I20260809 13:14:19.606683 9646 ts_tablet_manager.cc:1428] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 13:14:19.607050 9645 raft_consensus.cc:348] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.607394 9645 raft_consensus.cc:374] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:19.607542 9645 raft_consensus.cc:729] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75c9265a15a41b9b3f3949ba0a28115, State: Initialized, Role: FOLLOWER
I20260809 13:14:19.608100 9645 consensus_queue.cc:260] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.609424 9424 heartbeater.cc:503] Master 127.10.181.190:40131 was elected leader, sending a full tablet report...
I20260809 13:14:19.609819 9645 ts_tablet_manager.cc:1428] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260809 13:14:19.708544 9234 tablet.cc:2369] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:19.713259 9425 tablet.cc:2369] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 13:14:19.737995 9623 tablet.cc:2369] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:14:19.771917 9653 raft_consensus.cc:482] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:19.772218 9653 raft_consensus.cc:504] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.773910 9653 leader_election.cc:288] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19c0fd5a10cc4b938cff0282df66bb0b (127.10.181.129:41425), a75c9265a15a41b9b3f3949ba0a28115 (127.10.181.130:45259)
I20260809 13:14:19.779518 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.774269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:41425 (local address 127.10.181.131:57033)
0809 13:14:19.774484 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.774506 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.774637 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.775205 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.775210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.775251 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:19.775410 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.775418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.776406 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.776410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.777714 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.777727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.778808 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.778814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.778990 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.779016 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.779098 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.779222 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 13:14:19.779517 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.774323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:57033 (local address 127.10.181.129:41425)
0809 13:14:19.774517 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.774522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.774563 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:19.774820 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.774827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.774908 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.775022 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:19.775556 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.776264 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.777878 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.778658 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.778811 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.778884 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.779143 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.779267 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:14:19.780220 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.774934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:38401 (local address 127.10.181.130:45259)
0809 13:14:19.775137 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.775142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.775435 (+ 293us) server_negotiation.cc:407] Connection header received
0809 13:14:19.775598 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.775602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.775662 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.775734 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:19.776246 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.776934 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.778595 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.779339 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.779488 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.779558 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.779867 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.779985 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:14:19.780265 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.774668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:45259 (local address 127.10.181.131:38401)
0809 13:14:19.775335 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:19.775349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:19.775466 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:19.775759 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:19.775763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:19.775788 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:19.776099 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.776108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.777084 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.777088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.778435 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:19.778447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.779549 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.779555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:19.779705 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.779735 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:14:19.779824 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 13:14:19.779943 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":394,"threads_started":1}
I20260809 13:14:19.780611 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75955e306fd043be96ae89762b9f47ac" candidate_uuid: "e4068d80ac73421488bbd90709a14536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" is_pre_election: true
I20260809 13:14:19.781153 9168 raft_consensus.cc:2454] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4068d80ac73421488bbd90709a14536 in term 0.
I20260809 13:14:19.781288 9360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75955e306fd043be96ae89762b9f47ac" candidate_uuid: "e4068d80ac73421488bbd90709a14536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75c9265a15a41b9b3f3949ba0a28115" is_pre_election: true
I20260809 13:14:19.781826 9360 raft_consensus.cc:2454] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4068d80ac73421488bbd90709a14536 in term 0.
I20260809 13:14:19.781867 9460 leader_election.cc:302] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19c0fd5a10cc4b938cff0282df66bb0b, e4068d80ac73421488bbd90709a14536; no voters:
I20260809 13:14:19.782212 9653 raft_consensus.cc:2781] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:19.782344 9653 raft_consensus.cc:482] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:19.782450 9653 raft_consensus.cc:3037] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:19.784668 9653 raft_consensus.cc:504] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.785218 9653 leader_election.cc:288] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [CANDIDATE]: Term 1 election: Requested vote from peers 19c0fd5a10cc4b938cff0282df66bb0b (127.10.181.129:41425), a75c9265a15a41b9b3f3949ba0a28115 (127.10.181.130:45259)
I20260809 13:14:19.785643 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75955e306fd043be96ae89762b9f47ac" candidate_uuid: "e4068d80ac73421488bbd90709a14536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c0fd5a10cc4b938cff0282df66bb0b"
I20260809 13:14:19.785746 9360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75955e306fd043be96ae89762b9f47ac" candidate_uuid: "e4068d80ac73421488bbd90709a14536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75c9265a15a41b9b3f3949ba0a28115"
I20260809 13:14:19.785831 9168 raft_consensus.cc:3037] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:19.785976 9360 raft_consensus.cc:3037] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:19.788348 9168 raft_consensus.cc:2454] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4068d80ac73421488bbd90709a14536 in term 1.
I20260809 13:14:19.788421 9360 raft_consensus.cc:2454] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4068d80ac73421488bbd90709a14536 in term 1.
I20260809 13:14:19.788735 9460 leader_election.cc:302] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19c0fd5a10cc4b938cff0282df66bb0b, e4068d80ac73421488bbd90709a14536; no voters:
I20260809 13:14:19.789016 9653 raft_consensus.cc:2781] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:19.789464 9653 raft_consensus.cc:686] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 1 LEADER]: Becoming Leader. State: Replica: e4068d80ac73421488bbd90709a14536, State: Running, Role: LEADER
I20260809 13:14:19.789986 9653 consensus_queue.cc:237] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } }
I20260809 13:14:19.794332 8934 catalog_manager.cc:5495] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4068d80ac73421488bbd90709a14536 (127.10.181.131). New cstate: current_term: 1 leader_uuid: "e4068d80ac73421488bbd90709a14536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4068d80ac73421488bbd90709a14536" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 38729 } health_report { overall_health: HEALTHY } } }
I20260809 13:14:19.855978 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.850563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.10.181.131:38729)
0809 13:14:19.850760 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.850766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.850920 (+ 154us) server_negotiation.cc:407] Connection header received
0809 13:14:19.851156 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.851160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.851253 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.851386 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:19.851784 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.852519 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.853911 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.854310 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.854421 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.854907 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:19.855405 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:19.855541 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.855651 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.855735 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:14:19.863962 9360 raft_consensus.cc:1261] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 1 FOLLOWER]: Refusing update from remote peer e4068d80ac73421488bbd90709a14536: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:19.863993 9168 raft_consensus.cc:1261] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 1 FOLLOWER]: Refusing update from remote peer e4068d80ac73421488bbd90709a14536: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:19.864840 9653 consensus_queue.cc:1035] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19c0fd5a10cc4b938cff0282df66bb0b" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:19.865136 9658 consensus_queue.cc:1035] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75c9265a15a41b9b3f3949ba0a28115" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 45259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:19.888520 8933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:19.888785 8933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:19.892370 8933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f163c5eacca54adcaca7e292813be5ca: Sending DeleteTablet for 3 replicas of tablet 75955e306fd043be96ae89762b9f47ac
I20260809 13:14:19.893643 9534 tablet_service.cc:1514] Processing DeleteTablet for tablet 75955e306fd043be96ae89762b9f47ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:34032
I20260809 13:14:19.893643 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 75955e306fd043be96ae89762b9f47ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:43442
I20260809 13:14:19.893738 9331 tablet_service.cc:1514] Processing DeleteTablet for tablet 75955e306fd043be96ae89762b9f47ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:19 UTC) from {username='kudu', principal='kudu/127.10.181.190@KRBTEST.COM'} at 127.0.0.1:42884
I20260809 13:14:19.894577 9681 tablet_replica.cc:330] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: stopping tablet replica
I20260809 13:14:19.894974 9681 raft_consensus.cc:2229] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:19.895195 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 9032
I20260809 13:14:19.895756 9682 tablet_replica.cc:330] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: stopping tablet replica
I20260809 13:14:19.896126 9682 raft_consensus.cc:2229] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:19.896541 9681 raft_consensus.cc:2258] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:19.896584 9682 raft_consensus.cc:2258] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:19.898798 9682 ts_tablet_manager.cc:1905] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:19.904757 9681 ts_tablet_manager.cc:1905] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:19.906432 9682 ts_tablet_manager.cc:1918] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:19.906633 9682 log.cc:1196] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-0/wal/wals/75955e306fd043be96ae89762b9f47ac
I20260809 13:14:19.907203 9682 ts_tablet_manager.cc:1939] T 75955e306fd043be96ae89762b9f47ac P 19c0fd5a10cc4b938cff0282df66bb0b: Deleting consensus metadata
I20260809 13:14:19.908989 8915 catalog_manager.cc:4841] TS 19c0fd5a10cc4b938cff0282df66bb0b (127.10.181.129:41425): tablet 75955e306fd043be96ae89762b9f47ac (table test-table [id=d67bf274008d4746a014dcbe311fe834]) successfully deleted
I20260809 13:14:19.910037 9683 tablet_replica.cc:330] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: stopping tablet replica
I20260809 13:14:19.910480 9683 raft_consensus.cc:2229] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:19.911034 9683 raft_consensus.cc:2258] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:19.912362 9681 ts_tablet_manager.cc:1918] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:19.912628 9681 log.cc:1196] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/75955e306fd043be96ae89762b9f47ac
I20260809 13:14:19.913223 9681 ts_tablet_manager.cc:1939] T 75955e306fd043be96ae89762b9f47ac P e4068d80ac73421488bbd90709a14536: Deleting consensus metadata
I20260809 13:14:19.913409 9683 ts_tablet_manager.cc:1905] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:19.914897 8916 catalog_manager.cc:4841] TS e4068d80ac73421488bbd90709a14536 (127.10.181.131:38729): tablet 75955e306fd043be96ae89762b9f47ac (table test-table [id=d67bf274008d4746a014dcbe311fe834]) successfully deleted
Aug 09 13:14:19 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.129@KRBTEST.COM
I20260809 13:14:19.919844 9683 ts_tablet_manager.cc:1918] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:19.919994 9683 log.cc:1196] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/75955e306fd043be96ae89762b9f47ac
I20260809 13:14:19.920390 9683 ts_tablet_manager.cc:1939] T 75955e306fd043be96ae89762b9f47ac P a75c9265a15a41b9b3f3949ba0a28115: Deleting consensus metadata
I20260809 13:14:19.921844 8915 catalog_manager.cc:4841] TS a75c9265a15a41b9b3f3949ba0a28115 (127.10.181.130:45259): tablet 75955e306fd043be96ae89762b9f47ac (table test-table [id=d67bf274008d4746a014dcbe311fe834]) successfully deleted
I20260809 13:14:19.926898 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:19.897223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.10.181.129:41425)
0809 13:14:19.897382 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:19.897388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:19.909725 (+ 12337us) server_negotiation.cc:407] Connection header received
0809 13:14:19.909992 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:19.909997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:19.910094 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:19.910169 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:19.912036 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.913101 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.914952 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:19.915572 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:19.915678 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:19.921144 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:19.921159 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:19.921174 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:19.921219 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:19.924146 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.924940 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.924946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.924949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.925081 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:19.925530 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:19.925533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:19.925535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:19.925871 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:19.926095 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:19.926450 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 13:14:19.926604 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:14:19.927455 9224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:20.078696 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 9032
I20260809 13:14:20.103420 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 9239
Aug 09 13:14:20 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.130@KRBTEST.COM
I20260809 13:14:20.119060 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:20.104281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.10.181.130:45259)
0809 13:14:20.104466 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:20.104471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:20.104489 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:14:20.104664 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:20.104668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:20.104734 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:20.104814 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:20.106396 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:20.107146 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:20.108381 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:20.108762 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:20.108872 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:20.113626 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:20.113642 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:20.113659 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:20.113703 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:20.116359 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:20.117167 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:20.117173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:20.117176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:20.117305 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:20.117683 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:20.117687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:20.117688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:20.118061 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:20.118288 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:20.118630 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 13:14:20.118794 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:14:20.119699 9417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:20.208806 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 9239
I20260809 13:14:20.235055 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 9430
Aug 09 13:14:20 dist-test-slave-42z9 krb5kdc[8871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.131@KRBTEST.COM
I20260809 13:14:20.251609 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:20.236001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.10.181.131:38729)
0809 13:14:20.236209 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:20.236214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:20.236428 (+ 214us) server_negotiation.cc:407] Connection header received
0809 13:14:20.236607 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:20.236611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:20.236677 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:20.236787 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:20.238167 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:20.239040 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:20.240283 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:20.240731 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:20.240844 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:20.245542 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:20.245555 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:20.245573 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:20.245611 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:20.248567 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:20.249461 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:20.249472 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:20.249476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:20.249649 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:20.250062 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:20.250065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:20.250067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:20.250456 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:20.250690 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:20.251037 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:14:20.251238 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 13:14:20.252475 9615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:20.380175 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 9430
I20260809 13:14:20.406032 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 8890
I20260809 13:14:20.407110 8974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:20.515657 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 8890
2026-08-09T13:14:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4612 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T13:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:20Z Disabled control of system clock
I20260809 13:14:20.611698 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39331
--webserver_interface=127.10.181.190
--webserver_port=0
--builtin_ntp_servers=127.10.181.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:20.941884 9704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:20.942143 9704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:20.942225 9704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:20.949119 9704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:20.949213 9704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:20.949255 9704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:20.949309 9704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:20.949352 9704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:20.958097 9704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:20.959318 9704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:20.961004 9704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:20.971825 9711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:20.972291 9712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:20.972402 9716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:20.974205 9704 server_base.cc:1034] running on GCE node
I20260809 13:14:20.974918 9704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:20.976184 9704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:20.977406 9704 hybrid_clock.cc:648] HybridClock initialized: now 1786281260977344 us; error 42 us; skew 500 ppm
I20260809 13:14:20.977799 9704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:20.979962 9704 webserver.cc:460] Webserver started at http://127.10.181.190:37935/ using document root <none> and password file <none>
I20260809 13:14:20.980598 9704 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:20.980705 9704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:20.981015 9704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:20.983986 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "b1e3194eddf9462396825f36781f1d8a"
format_stamp: "Formatted at 2026-08-09 13:14:20 on dist-test-slave-42z9"
server_key: "e7b522fbc5ceee49d9463f00ca144083"
server_key_iv: "a8264567bca1e60123d51ecc5b5a66ee"
server_key_version: "encryptionkey@0"
I20260809 13:14:20.984779 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "b1e3194eddf9462396825f36781f1d8a"
format_stamp: "Formatted at 2026-08-09 13:14:20 on dist-test-slave-42z9"
server_key: "e7b522fbc5ceee49d9463f00ca144083"
server_key_iv: "a8264567bca1e60123d51ecc5b5a66ee"
server_key_version: "encryptionkey@0"
I20260809 13:14:20.989768 9704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 13:14:20.993043 9725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:20.994357 9704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:20.994520 9704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "b1e3194eddf9462396825f36781f1d8a"
format_stamp: "Formatted at 2026-08-09 13:14:20 on dist-test-slave-42z9"
server_key: "e7b522fbc5ceee49d9463f00ca144083"
server_key_iv: "a8264567bca1e60123d51ecc5b5a66ee"
server_key_version: "encryptionkey@0"
I20260809 13:14:20.994731 9704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:21.023386 9704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:21.024245 9704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:21.024523 9704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:21.041569 9704 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39331
I20260809 13:14:21.041594 9799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39331 every 8 connection(s)
I20260809 13:14:21.042907 9704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:21.046588 9800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:21.047672 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 9704
I20260809 13:14:21.047813 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:21.048259 10966 external_mini_cluster.cc:1428] Setting key cd9f08d1efe4c463f36c152ae03e6aa9
I20260809 13:14:21.060885 9800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: Bootstrap starting.
I20260809 13:14:21.062685 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:21.050207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.10.181.190:39331)
0809 13:14:21.051277 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:21.051294 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:21.051353 (+ 59us) server_negotiation.cc:407] Connection header received
0809 13:14:21.053030 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:21.053057 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:21.053658 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:21.054097 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:21.054592 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.056388 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.057685 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.058754 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.058960 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:21.060009 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:21.060020 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:21.060036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:21.060695 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:21.060800 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:21.060826 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:21.060967 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:21.061126 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 13:14:21.061428 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":308,"threads_started":1}
I20260809 13:14:21.064714 9800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:21.065979 9800 log.cc:824] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:21.068451 9800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: No bootstrap required, opened a new log
I20260809 13:14:21.076063 9800 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } }
I20260809 13:14:21.076381 9800 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:21.076469 9800 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e3194eddf9462396825f36781f1d8a, State: Initialized, Role: FOLLOWER
I20260809 13:14:21.076921 9800 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } }
I20260809 13:14:21.077121 9800 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:21.077214 9800 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:21.077327 9800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:21.079326 9800 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } }
I20260809 13:14:21.079778 9800 leader_election.cc:302] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e3194eddf9462396825f36781f1d8a; no voters:
I20260809 13:14:21.080262 9800 leader_election.cc:288] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:21.080376 9805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:21.081022 9805 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [term 1 LEADER]: Becoming Leader. State: Replica: b1e3194eddf9462396825f36781f1d8a, State: Running, Role: LEADER
I20260809 13:14:21.081558 9805 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } }
I20260809 13:14:21.082187 9800 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:21.084352 9807 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e3194eddf9462396825f36781f1d8a. Latest consensus state: current_term: 1 leader_uuid: "b1e3194eddf9462396825f36781f1d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } } }
I20260809 13:14:21.084372 9806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1e3194eddf9462396825f36781f1d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e3194eddf9462396825f36781f1d8a" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39331 } } }
I20260809 13:14:21.084807 9806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:21.084811 9807 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:21.086366 9813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:21.091554 9813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:21.100841 9813 catalog_manager.cc:1347] Generated new cluster ID: 1d82620979d84dab8919121a3760a9f5
I20260809 13:14:21.100975 9813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:21.120313 9813 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:21.121974 9813 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:21.127869 9813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: Generated new TSK 0
I20260809 13:14:21.128609 9813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:21.135458 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.129:0
--local_ip_for_outbound_sockets=127.10.181.129
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--builtin_ntp_servers=127.10.181.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 13:14:21.478520 9830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:21.478823 9830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:21.478960 9830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:21.486289 9830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:21.486394 9830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:21.486533 9830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.129
I20260809 13:14:21.495543 9830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.181.129
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.129
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:21.496837 9830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:21.498721 9830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:21.511005 9837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:21.511185 9836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:21.511065 9843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:21.512017 9830 server_base.cc:1034] running on GCE node
I20260809 13:14:21.512610 9830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:21.513517 9830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:21.514726 9830 hybrid_clock.cc:648] HybridClock initialized: now 1786281261514677 us; error 43 us; skew 500 ppm
I20260809 13:14:21.515033 9830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:21.517181 9830 webserver.cc:460] Webserver started at http://127.10.181.129:32861/ using document root <none> and password file <none>
I20260809 13:14:21.517910 9830 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:21.518013 9830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:21.518309 9830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:21.521149 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data/instance:
uuid: "3dd6b261fbd3452ab6e1943999edd25f"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "4d5ba6ceb17f2f4f99e4d98538d91fd4"
server_key_iv: "1ba36af1462a1b6518ec73ff3103062e"
server_key_version: "encryptionkey@0"
I20260809 13:14:21.521952 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance:
uuid: "3dd6b261fbd3452ab6e1943999edd25f"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "4d5ba6ceb17f2f4f99e4d98538d91fd4"
server_key_iv: "1ba36af1462a1b6518ec73ff3103062e"
server_key_version: "encryptionkey@0"
I20260809 13:14:21.526757 9830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 13:14:21.530067 9854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:21.531275 9830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:21.531430 9830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal
uuid: "3dd6b261fbd3452ab6e1943999edd25f"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "4d5ba6ceb17f2f4f99e4d98538d91fd4"
server_key_iv: "1ba36af1462a1b6518ec73ff3103062e"
server_key_version: "encryptionkey@0"
I20260809 13:14:21.531629 9830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:21.548324 9830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:21.549084 9830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:21.549363 9830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:21.550343 9830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:21.551733 9830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:21.551833 9830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:21.551942 9830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:21.552011 9830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:21.582403 9830 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.129:46873
I20260809 13:14:21.582458 10020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.129:46873 every 8 connection(s)
I20260809 13:14:21.583777 9830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/data/info.pb
I20260809 13:14:21.591560 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 9830
I20260809 13:14:21.591734 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-0/wal/instance
I20260809 13:14:21.592229 10966 external_mini_cluster.cc:1428] Setting key 67718ce49b550565b3cef3af12f335fe
I20260809 13:14:21.595132 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.130:0
--local_ip_for_outbound_sockets=127.10.181.130
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--builtin_ntp_servers=127.10.181.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:21.596665 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:21.586290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.129:57397 (local address 127.10.181.190:39331)
0809 13:14:21.586707 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:21.586712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:21.587887 (+ 1175us) server_negotiation.cc:407] Connection header received
0809 13:14:21.589689 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:21.589695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:21.589776 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:21.589874 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:21.590928 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.591906 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.593790 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.594317 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.594441 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:21.595787 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:21.595794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:21.595798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:21.595820 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:21.595875 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:21.595892 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:21.595985 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:21.596235 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 13:14:21.596332 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":212,"threads_started":1}
I20260809 13:14:21.597566 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:21.586804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39331 (local address 127.10.181.129:57397)
0809 13:14:21.587724 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:21.587759 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:21.589401 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:21.590121 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:21.590130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:21.590212 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:21.590763 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:21.590779 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.592065 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.592069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:21.593650 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:21.593664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:21.594541 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:21.594546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:21.594748 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:21.595472 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:21.595474 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:21.595515 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:21.595593 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:21.596010 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:21.596022 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:21.596031 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:14:21.596197 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 13:14:21.596490 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":288,"threads_started":1}
I20260809 13:14:21.599097 10021 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39331
I20260809 13:14:21.599400 10021 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:21.600139 10021 heartbeater.cc:511] Master 127.10.181.190:39331 requested a full tablet report, sending...
I20260809 13:14:21.602169 9748 ts_manager.cc:194] Registered new tserver with Master: 3dd6b261fbd3452ab6e1943999edd25f (127.10.181.129:46873)
I20260809 13:14:21.603606 9748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.129:57397
W20260809 13:14:21.953639 10026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:21.953907 10026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:21.954030 10026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:21.961125 10026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:21.961225 10026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:21.961369 10026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.130
I20260809 13:14:21.973026 10026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.181.130
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.130
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:21.974370 10026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:21.976567 10026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:21.991255 10034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:21.991489 10036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:21.991887 10043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:21.992297 10026 server_base.cc:1034] running on GCE node
I20260809 13:14:21.992889 10026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:21.993860 10026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:21.995082 10026 hybrid_clock.cc:648] HybridClock initialized: now 1786281261995013 us; error 59 us; skew 500 ppm
I20260809 13:14:21.995394 10026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:21.997716 10026 webserver.cc:460] Webserver started at http://127.10.181.130:36351/ using document root <none> and password file <none>
I20260809 13:14:21.998454 10026 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:21.998562 10026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:21.998857 10026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:22.002099 10026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data/instance:
uuid: "10439ea4f4ee4440ac50d31dc130c4ae"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "e555c1b077fa5f47b5683639674addfc"
server_key_iv: "8a14adbe00064f6c84c883476839deb2"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.002894 10026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance:
uuid: "10439ea4f4ee4440ac50d31dc130c4ae"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "e555c1b077fa5f47b5683639674addfc"
server_key_iv: "8a14adbe00064f6c84c883476839deb2"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.008046 10026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 13:14:22.011513 10053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.012823 10026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:14:22.012972 10026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal
uuid: "10439ea4f4ee4440ac50d31dc130c4ae"
format_stamp: "Formatted at 2026-08-09 13:14:21 on dist-test-slave-42z9"
server_key: "e555c1b077fa5f47b5683639674addfc"
server_key_iv: "8a14adbe00064f6c84c883476839deb2"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.013165 10026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:22.034531 10026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:22.035380 10026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:22.035693 10026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:22.036794 10026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:22.038383 10026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:22.038484 10026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.038594 10026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:22.038671 10026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.077023 10026 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.130:41179
I20260809 13:14:22.077142 10228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.130:41179 every 8 connection(s)
I20260809 13:14:22.078372 10026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/data/info.pb
I20260809 13:14:22.082510 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10026
I20260809 13:14:22.082683 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal/instance
I20260809 13:14:22.083252 10966 external_mini_cluster.cc:1428] Setting key cf7feb9a5dd0756d9f421c134d60f7d6
I20260809 13:14:22.086043 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.181.131:0
--local_ip_for_outbound_sockets=127.10.181.131
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--builtin_ntp_servers=127.10.181.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 13:14:22.094259 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.081490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.130:39559 (local address 127.10.181.190:39331)
0809 13:14:22.081698 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.081705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.083495 (+ 1790us) server_negotiation.cc:407] Connection header received
0809 13:14:22.086113 (+ 2618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.086118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.086204 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.086313 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:22.087802 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.088839 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.091224 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.091752 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.091876 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.093463 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:22.093470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:22.093474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:22.093497 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:22.093535 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:22.093550 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:22.093639 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.093875 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.093973 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 13:14:22.095693 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.081964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39331 (local address 127.10.181.130:39559)
0809 13:14:22.083288 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.083337 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.085767 (+ 2430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.086575 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.086587 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.086709 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:22.087614 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.087641 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.089007 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.089013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.091061 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.091077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.091991 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.091998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.092287 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.093051 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:22.093055 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:22.093112 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:22.093221 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:22.093647 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:22.093659 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:22.093668 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.093856 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.094269 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":402,"threads_started":1}
I20260809 13:14:22.097261 10229 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39331
I20260809 13:14:22.097625 10229 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:22.098590 10229 heartbeater.cc:511] Master 127.10.181.190:39331 requested a full tablet report, sending...
I20260809 13:14:22.100167 9748 ts_manager.cc:194] Registered new tserver with Master: 10439ea4f4ee4440ac50d31dc130c4ae (127.10.181.130:41179)
I20260809 13:14:22.101020 9748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.130:39559
W20260809 13:14:22.462224 10232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:22.462507 10232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:22.462602 10232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:22.469707 10232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:22.469812 10232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:22.469924 10232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.181.131
I20260809 13:14:22.478787 10232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.181.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.181.131
--webserver_port=0
--tserver_master_addrs=127.10.181.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.181.131
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:22.480062 10232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:22.481815 10232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:22.494366 10241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:22.494390 10247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:22.494320 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:22.495355 10232 server_base.cc:1034] running on GCE node
I20260809 13:14:22.495940 10232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:22.496883 10232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:22.498116 10232 hybrid_clock.cc:648] HybridClock initialized: now 1786281262498062 us; error 26 us; skew 500 ppm
I20260809 13:14:22.498515 10232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 13:14:22.503659 10232 webserver.cc:460] Webserver started at http://127.10.181.131:44031/ using document root <none> and password file <none>
I20260809 13:14:22.504362 10232 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:22.504454 10232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:22.504740 10232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:22.507776 10232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data/instance:
uuid: "c53721188f9a46e98e2ef5410a24e82d"
format_stamp: "Formatted at 2026-08-09 13:14:22 on dist-test-slave-42z9"
server_key: "6defc4813940e8836b96b94a676793c5"
server_key_iv: "7880bd519f32d90f71d3a0f71453fe0a"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.508559 10232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance:
uuid: "c53721188f9a46e98e2ef5410a24e82d"
format_stamp: "Formatted at 2026-08-09 13:14:22 on dist-test-slave-42z9"
server_key: "6defc4813940e8836b96b94a676793c5"
server_key_iv: "7880bd519f32d90f71d3a0f71453fe0a"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.513728 10232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 13:14:22.517120 10253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.518463 10232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:22.518605 10232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal
uuid: "c53721188f9a46e98e2ef5410a24e82d"
format_stamp: "Formatted at 2026-08-09 13:14:22 on dist-test-slave-42z9"
server_key: "6defc4813940e8836b96b94a676793c5"
server_key_iv: "7880bd519f32d90f71d3a0f71453fe0a"
server_key_version: "encryptionkey@0"
I20260809 13:14:22.518790 10232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:22.535307 10232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:22.536029 10232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:22.536338 10232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:22.537236 10232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 13:14:22.538658 10232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 13:14:22.538748 10232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.538825 10232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 13:14:22.538878 10232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:22.568356 10232 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.131:35283
I20260809 13:14:22.568406 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.131:35283 every 8 connection(s)
I20260809 13:14:22.569629 10232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/data/info.pb
I20260809 13:14:22.575088 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10232
I20260809 13:14:22.575263 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal/instance
I20260809 13:14:22.575695 10966 external_mini_cluster.cc:1428] Setting key 47c5eeab136ac2a941bc93604d4db9ef
I20260809 13:14:22.581986 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.572071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:51523 (local address 127.10.181.190:39331)
0809 13:14:22.572257 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.572261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.573620 (+ 1359us) server_negotiation.cc:407] Connection header received
0809 13:14:22.575718 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.575723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.575811 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.575904 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:22.577124 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.577979 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.579482 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.579876 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.579984 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.581170 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:22.581175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:22.581178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:22.581195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:22.581226 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:22.581237 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:22.581308 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.581541 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.581649 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:14:22.583274 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.572557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39331 (local address 127.10.181.131:51523)
0809 13:14:22.573431 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.573502 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.575452 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.576161 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.576172 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.576278 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:14:22.576926 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.576954 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.578138 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.578144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.579332 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.579345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.580092 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.580100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.580299 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.580875 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 13:14:22.580878 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:14:22.580921 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 13:14:22.580989 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:14:22.581324 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:14:22.581335 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:14:22.581344 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.581515 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.581907 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":267,"threads_started":1}
I20260809 13:14:22.584508 10412 heartbeater.cc:347] Connected to a master server at 127.10.181.190:39331
I20260809 13:14:22.584791 10412 heartbeater.cc:464] Registering TS with master...
I20260809 13:14:22.585570 10412 heartbeater.cc:511] Master 127.10.181.190:39331 requested a full tablet report, sending...
I20260809 13:14:22.586977 9748 ts_manager.cc:194] Registered new tserver with Master: c53721188f9a46e98e2ef5410a24e82d (127.10.181.131:35283)
I20260809 13:14:22.587749 9748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.181.131:51523
I20260809 13:14:22.590641 10966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 13:14:22.599692 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.594428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.10.181.190:39331)
0809 13:14:22.594671 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.594677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.595022 (+ 345us) server_negotiation.cc:407] Connection header received
0809 13:14:22.595158 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.595161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.595231 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.595309 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:22.595685 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.596321 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.597720 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.598094 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.598189 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.599087 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:22.599092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:22.599095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:22.599111 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:22.599146 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:22.599158 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:22.599224 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.599306 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.599388 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:14:22.606473 10021 heartbeater.cc:503] Master 127.10.181.190:39331 was elected leader, sending a full tablet report...
I20260809 13:14:22.603003 9748 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 13:14:22.610399 9748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 13:14:22.634382 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.622058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35650)
0809 13:14:22.622758 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.622806 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.623047 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.624354 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.624363 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.624399 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:22.625971 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.625988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.627975 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.627981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.631513 (+ 3532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.631524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.633344 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.633352 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.633639 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.633707 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.633817 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.634017 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":49,"thread_start_us":814,"threads_started":1}
I20260809 13:14:22.634361 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.622636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.10.181.129:46873)
0809 13:14:22.623476 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.623484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.623516 (+ 32us) server_negotiation.cc:407] Connection header received
0809 13:14:22.623637 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.623645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.623881 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.624065 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:22.626178 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.627803 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.631676 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.632757 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.632916 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.633164 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.633878 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.634049 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":735,"threads_started":1}
I20260809 13:14:22.634497 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.621864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54494 (local address 127.10.181.130:41179)
0809 13:14:22.622990 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.623010 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.623985 (+ 975us) server_negotiation.cc:407] Connection header received
0809 13:14:22.624289 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.624297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.624522 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.624697 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:22.625728 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.627374 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.631382 (+ 4008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.632776 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.632947 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.633177 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.633949 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.634134 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":860,"threads_started":1}
I20260809 13:14:22.635162 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.622378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41179 (local address 127.0.0.1:54494)
0809 13:14:22.623872 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.623889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.624093 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.625009 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.625014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.625047 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:22.625499 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.625512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.627579 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.627584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.631130 (+ 3546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.631149 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.633401 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.633410 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.633723 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.633758 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.634698 (+ 940us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.634828 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":37,"thread_start_us":1273,"threads_started":1}
I20260809 13:14:22.637538 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.623999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.131:35283 (local address 127.0.0.1:42774)
0809 13:14:22.630077 (+ 6078us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.630098 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.630211 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.630687 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.630691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.630714 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:22.631123 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.631136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.632984 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.632988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.635142 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.635162 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.636770 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.636776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.636939 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.636969 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.637049 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.637174 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5958,"mutex_wait_us":2,"thread_start_us":5792,"threads_started":1}
I20260809 13:14:22.637653 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.624360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.10.181.131:35283)
0809 13:14:22.629854 (+ 5494us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.629862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.630184 (+ 322us) server_negotiation.cc:407] Connection header received
0809 13:14:22.630333 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.630338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.630505 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.630670 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:22.631312 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.632834 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.635382 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.636591 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.636765 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.636959 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.637113 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.637262 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5318,"thread_start_us":5144,"threads_started":1}
I20260809 13:14:22.638918 9929 tablet_service.cc:1467] Processing CreateTablet for tablet e068201774e74027895b4cd78948a41e (DEFAULT_TABLE table=test-table [id=ba21cdb587284a309c959cc56ae671c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:22.638918 10122 tablet_service.cc:1467] Processing CreateTablet for tablet e068201774e74027895b4cd78948a41e (DEFAULT_TABLE table=test-table [id=ba21cdb587284a309c959cc56ae671c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:22.640377 9929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068201774e74027895b4cd78948a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:22.640390 10122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068201774e74027895b4cd78948a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:22.640300 10323 tablet_service.cc:1467] Processing CreateTablet for tablet e068201774e74027895b4cd78948a41e (DEFAULT_TABLE table=test-table [id=ba21cdb587284a309c959cc56ae671c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 13:14:22.641357 10323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e068201774e74027895b4cd78948a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:22.652006 10446 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Bootstrap starting.
I20260809 13:14:22.654439 10449 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Bootstrap starting.
I20260809 13:14:22.656639 10446 tablet_bootstrap.cc:654] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:22.657927 10449 tablet_bootstrap.cc:654] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:22.657919 10446 log.cc:824] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:22.658330 10448 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Bootstrap starting.
I20260809 13:14:22.659144 10449 log.cc:824] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:22.661188 10448 tablet_bootstrap.cc:654] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:22.661655 10446 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: No bootstrap required, opened a new log
I20260809 13:14:22.661901 10446 ts_tablet_manager.cc:1397] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 13:14:22.662379 10448 log.cc:824] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:22.664034 10449 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: No bootstrap required, opened a new log
I20260809 13:14:22.664304 10449 ts_tablet_manager.cc:1397] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 13:14:22.664959 10448 tablet_bootstrap.cc:492] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: No bootstrap required, opened a new log
I20260809 13:14:22.665190 10448 ts_tablet_manager.cc:1397] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 13:14:22.671707 10448 raft_consensus.cc:348] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.672039 10448 raft_consensus.cc:374] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:22.672135 10448 raft_consensus.cc:729] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dd6b261fbd3452ab6e1943999edd25f, State: Initialized, Role: FOLLOWER
I20260809 13:14:22.672165 10446 raft_consensus.cc:348] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.672567 10446 raft_consensus.cc:374] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:22.672670 10446 raft_consensus.cc:729] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53721188f9a46e98e2ef5410a24e82d, State: Initialized, Role: FOLLOWER
I20260809 13:14:22.672715 10448 consensus_queue.cc:260] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.673230 10446 consensus_queue.cc:260] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.674494 10448 ts_tablet_manager.cc:1428] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20260809 13:14:22.674278 10449 raft_consensus.cc:348] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.674680 10449 raft_consensus.cc:374] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:22.674765 10449 raft_consensus.cc:729] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10439ea4f4ee4440ac50d31dc130c4ae, State: Initialized, Role: FOLLOWER
I20260809 13:14:22.675446 10449 consensus_queue.cc:260] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.675851 10412 heartbeater.cc:503] Master 127.10.181.190:39331 was elected leader, sending a full tablet report...
I20260809 13:14:22.676427 10446 ts_tablet_manager.cc:1428] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 13:14:22.677284 10229 heartbeater.cc:503] Master 127.10.181.190:39331 was elected leader, sending a full tablet report...
I20260809 13:14:22.679185 10449 ts_tablet_manager.cc:1428] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260809 13:14:22.696481 10454 raft_consensus.cc:482] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 13:14:22.696740 10454 raft_consensus.cc:504] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.698518 10454 leader_election.cc:288] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10439ea4f4ee4440ac50d31dc130c4ae (127.10.181.130:41179), 3dd6b261fbd3452ab6e1943999edd25f (127.10.181.129:46873)
I20260809 13:14:22.704634 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.699091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:38961 (local address 127.10.181.129:46873)
0809 13:14:22.699305 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.699311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.699329 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:14:22.699420 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.699425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.699511 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.699599 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:22.700067 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.700974 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.702810 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.703795 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.703909 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.703959 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.704261 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.704344 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:14:22.704659 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.698847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.10.181.131:38961)
0809 13:14:22.699142 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.699165 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.699297 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.699685 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.699689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.699718 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:22.699911 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.699922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.701112 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.701118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.702654 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.702666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.703932 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.703938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.704112 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.704148 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.704247 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.704376 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 13:14:22.705219 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.698796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.181.131:35935 (local address 127.10.181.130:41179)
0809 13:14:22.698982 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.698988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.699793 (+ 805us) server_negotiation.cc:407] Connection header received
0809 13:14:22.699937 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.699942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.700028 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.700108 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 13:14:22.700691 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.701538 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.703387 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.704301 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.704439 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.704503 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.704841 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.704968 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:14:22.705821 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.698892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.130:41179 (local address 127.10.181.131:35935)
0809 13:14:22.699691 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.699705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 13:14:22.699818 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:14:22.700157 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:14:22.700161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:14:22.700187 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 13:14:22.700538 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.700549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.701690 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.701694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.703204 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:14:22.703220 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.704462 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.704469 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:14:22.704641 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.704678 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 13:14:22.705418 (+ 740us) client_negotiation.cc:240] Negotiation successful
0809 13:14:22.705561 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":491,"threads_started":1}
I20260809 13:14:22.705750 9958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068201774e74027895b4cd78948a41e" candidate_uuid: "c53721188f9a46e98e2ef5410a24e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd6b261fbd3452ab6e1943999edd25f" is_pre_election: true
I20260809 13:14:22.706310 9958 raft_consensus.cc:2454] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c53721188f9a46e98e2ef5410a24e82d in term 0.
I20260809 13:14:22.706799 10154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068201774e74027895b4cd78948a41e" candidate_uuid: "c53721188f9a46e98e2ef5410a24e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" is_pre_election: true
I20260809 13:14:22.706991 10255 leader_election.cc:302] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dd6b261fbd3452ab6e1943999edd25f, c53721188f9a46e98e2ef5410a24e82d; no voters:
I20260809 13:14:22.707274 10154 raft_consensus.cc:2454] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c53721188f9a46e98e2ef5410a24e82d in term 0.
I20260809 13:14:22.707340 10454 raft_consensus.cc:2781] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 13:14:22.707453 10454 raft_consensus.cc:482] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 13:14:22.707551 10454 raft_consensus.cc:3037] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:22.709774 10454 raft_consensus.cc:504] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.710278 10454 leader_election.cc:288] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [CANDIDATE]: Term 1 election: Requested vote from peers 10439ea4f4ee4440ac50d31dc130c4ae (127.10.181.130:41179), 3dd6b261fbd3452ab6e1943999edd25f (127.10.181.129:46873)
I20260809 13:14:22.710681 10154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068201774e74027895b4cd78948a41e" candidate_uuid: "c53721188f9a46e98e2ef5410a24e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10439ea4f4ee4440ac50d31dc130c4ae"
I20260809 13:14:22.710737 9958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e068201774e74027895b4cd78948a41e" candidate_uuid: "c53721188f9a46e98e2ef5410a24e82d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd6b261fbd3452ab6e1943999edd25f"
I20260809 13:14:22.710839 10154 raft_consensus.cc:3037] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:22.710927 9958 raft_consensus.cc:3037] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:22.713008 10154 raft_consensus.cc:2454] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c53721188f9a46e98e2ef5410a24e82d in term 1.
I20260809 13:14:22.713086 9958 raft_consensus.cc:2454] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c53721188f9a46e98e2ef5410a24e82d in term 1.
I20260809 13:14:22.713384 10257 leader_election.cc:302] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10439ea4f4ee4440ac50d31dc130c4ae, c53721188f9a46e98e2ef5410a24e82d; no voters:
I20260809 13:14:22.713725 10454 raft_consensus.cc:2781] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:22.714146 10454 raft_consensus.cc:686] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 1 LEADER]: Becoming Leader. State: Replica: c53721188f9a46e98e2ef5410a24e82d, State: Running, Role: LEADER
I20260809 13:14:22.714645 10454 consensus_queue.cc:237] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } }
I20260809 13:14:22.719074 9747 catalog_manager.cc:5495] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d reported cstate change: term changed from 0 to 1, leader changed from <none> to c53721188f9a46e98e2ef5410a24e82d (127.10.181.131). New cstate: current_term: 1 leader_uuid: "c53721188f9a46e98e2ef5410a24e82d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c53721188f9a46e98e2ef5410a24e82d" member_type: VOTER last_known_addr { host: "127.10.181.131" port: 35283 } health_report { overall_health: HEALTHY } } }
I20260809 13:14:22.736801 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.731738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.10.181.131:35283)
0809 13:14:22.731936 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.731942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.732018 (+ 76us) server_negotiation.cc:407] Connection header received
0809 13:14:22.732193 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.732197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.732273 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.732394 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:14:22.732679 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.733387 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.734835 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.735279 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.735369 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.735815 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:14:22.736344 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:14:22.736451 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.736506 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.736567 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:14:22.744726 10154 raft_consensus.cc:1261] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 1 FOLLOWER]: Refusing update from remote peer c53721188f9a46e98e2ef5410a24e82d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:22.744726 9958 raft_consensus.cc:1261] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 1 FOLLOWER]: Refusing update from remote peer c53721188f9a46e98e2ef5410a24e82d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:14:22.745646 10459 consensus_queue.cc:1035] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dd6b261fbd3452ab6e1943999edd25f" member_type: VOTER last_known_addr { host: "127.10.181.129" port: 46873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:22.746004 10454 consensus_queue.cc:1035] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [LEADER]: Connected to new peer: Peer: permanent_uuid: "10439ea4f4ee4440ac50d31dc130c4ae" member_type: VOTER last_known_addr { host: "127.10.181.130" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:14:22.756737 10463 mvcc.cc:205] Tried to move back new op lower bound from 7316608052193005568 to 7316608052083187712. Current Snapshot: MvccSnapshot[applied={T|T < 7316608052193005568}]
I20260809 13:14:22.761188 10465 mvcc.cc:205] Tried to move back new op lower bound from 7316608052193005568 to 7316608052083187712. Current Snapshot: MvccSnapshot[applied={T|T < 7316608052193005568}]
I20260809 13:14:22.769210 9748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:22.769549 9748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 13:14:22.773083 9748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1e3194eddf9462396825f36781f1d8a: Sending DeleteTablet for 3 replicas of tablet e068201774e74027895b4cd78948a41e
I20260809 13:14:22.774159 9929 tablet_service.cc:1514] Processing DeleteTablet for tablet e068201774e74027895b4cd78948a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:22 UTC) from {username='slave'} at 127.0.0.1:35650
I20260809 13:14:22.774231 10323 tablet_service.cc:1514] Processing DeleteTablet for tablet e068201774e74027895b4cd78948a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:22 UTC) from {username='slave'} at 127.0.0.1:42774
I20260809 13:14:22.775053 10122 tablet_service.cc:1514] Processing DeleteTablet for tablet e068201774e74027895b4cd78948a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 13:14:22 UTC) from {username='slave'} at 127.0.0.1:54494
I20260809 13:14:22.775607 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 9830
I20260809 13:14:22.784040 10476 tablet_replica.cc:330] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: stopping tablet replica
I20260809 13:14:22.784495 10476 raft_consensus.cc:2229] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:22.784997 10476 raft_consensus.cc:2258] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:22.790015 10476 ts_tablet_manager.cc:1905] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:22.793859 10474 tablet_replica.cc:330] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: stopping tablet replica
I20260809 13:14:22.794143 10475 tablet_replica.cc:330] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: stopping tablet replica
I20260809 13:14:22.794319 10474 raft_consensus.cc:2229] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 13:14:22.794543 10475 raft_consensus.cc:2229] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 1 LEADER]: Raft consensus shutting down.
I20260809 13:14:22.794839 10474 raft_consensus.cc:2258] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:22.795322 10475 raft_consensus.cc:2258] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 13:14:22.797267 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.781677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.10.181.129:46873)
0809 13:14:22.781943 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.781949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.785806 (+ 3857us) server_negotiation.cc:407] Connection header received
0809 13:14:22.786502 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.786509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.786618 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.786695 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:22.788019 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.789300 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.791682 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.792410 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.792507 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.795583 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:22.795595 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:22.795612 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:22.796365 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:22.796425 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:22.796451 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:22.796543 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.796796 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.796910 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260809 13:14:22.797279 10474 ts_tablet_manager.cc:1905] T e068201774e74027895b4cd78948a41e P 3dd6b261fbd3452ab6e1943999edd25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:22.798439 10011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:22.799751 10475 ts_tablet_manager.cc:1905] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 13:14:22.799921 10476 ts_tablet_manager.cc:1918] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:22.800079 10476 log.cc:1196] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-1/wal/wals/e068201774e74027895b4cd78948a41e
I20260809 13:14:22.800622 10476 ts_tablet_manager.cc:1939] T e068201774e74027895b4cd78948a41e P 10439ea4f4ee4440ac50d31dc130c4ae: Deleting consensus metadata
I20260809 13:14:22.802418 9730 catalog_manager.cc:4841] TS 10439ea4f4ee4440ac50d31dc130c4ae (127.10.181.130:41179): tablet e068201774e74027895b4cd78948a41e (table test-table [id=ba21cdb587284a309c959cc56ae671c4]) successfully deleted
I20260809 13:14:22.807024 10475 ts_tablet_manager.cc:1918] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 13:14:22.807201 10475 log.cc:1196] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Deleting WAL directory at /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786280946828509-10966-0/minicluster-data/ts-2/wal/wals/e068201774e74027895b4cd78948a41e
I20260809 13:14:22.807731 10475 ts_tablet_manager.cc:1939] T e068201774e74027895b4cd78948a41e P c53721188f9a46e98e2ef5410a24e82d: Deleting consensus metadata
I20260809 13:14:22.809353 9730 catalog_manager.cc:4841] TS c53721188f9a46e98e2ef5410a24e82d (127.10.181.131:35283): tablet e068201774e74027895b4cd78948a41e (table test-table [id=ba21cdb587284a309c959cc56ae671c4]) successfully deleted
I20260809 13:14:22.889477 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 9830
I20260809 13:14:22.929001 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10026
W20260809 13:14:22.929031 9727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.181.129:46873 (error 108)
I20260809 13:14:22.931842 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.931436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35680)
0809 13:14:22.931597 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.931706 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:14:22.932143 9727 catalog_manager.cc:4584] TS 3dd6b261fbd3452ab6e1943999edd25f (127.10.181.129:46873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e068201774e74027895b4cd78948a41e: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
I20260809 13:14:22.937223 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.930038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.10.181.130:41179)
0809 13:14:22.930248 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:22.930254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:22.930373 (+ 119us) server_negotiation.cc:407] Connection header received
0809 13:14:22.930568 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:22.930574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:22.930708 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:22.930826 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:22.931504 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.932648 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.933917 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:22.934558 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:22.934693 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:22.935580 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:22.935591 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:22.935610 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:22.936394 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:22.936459 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:22.936485 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:22.936601 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:22.936758 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 13:14:22.936870 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 13:14:22.937997 10223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:22.985832 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.985197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35694)
0809 13:14:22.985495 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.985616 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 13:14:22.987082 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:22.986655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35700)
0809 13:14:22.986820 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:22.986929 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 13:14:23.026924 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10026
I20260809 13:14:23.048038 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.047349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35716)
0809 13:14:23.047698 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:23.047804 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 13:14:23.049368 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.048909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35720)
0809 13:14:23.049141 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:23.049226 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260809 13:14:23.063588 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10232
I20260809 13:14:23.071269 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.064384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.10.181.131:35283)
0809 13:14:23.064587 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:23.064593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:23.064701 (+ 108us) server_negotiation.cc:407] Connection header received
0809 13:14:23.064882 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:23.064886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:23.064969 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:23.065049 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:23.065575 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:23.066357 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:23.068164 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:23.068570 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:23.068658 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:23.069874 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:23.069886 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:23.069903 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:23.070527 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:23.070577 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:23.070597 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:23.070684 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:23.070851 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 13:14:23.070931 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:14:23.071903 10406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:23.130111 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.129608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35734)
0809 13:14:23.129809 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:23.129938 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 13:14:23.131299 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.130878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35736)
0809 13:14:23.131100 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:23.131187 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 13:14:23.196413 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10232
I20260809 13:14:23.225847 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 9704
I20260809 13:14:23.226892 9791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:23.333796 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:23.333134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.129:46873 (local address 127.0.0.1:35740)
0809 13:14:23.333357 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 13:14:23.333518 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.129:46873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 13:14:23.334033 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 9704
2026-08-09T13:14:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2819 ms)
[----------] 6 tests from EncryptionPolicyTest (22301 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:23 dist-test-slave-42z9 krb5kdc[10490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:23 dist-test-slave-42z9 krb5kdc[10490](info): set up 2 sockets
Aug 09 13:14:23 dist-test-slave-42z9 krb5kdc[10490](info): commencing operation
krb5kdc: starting...
W20260809 13:14:24.565121 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:24 dist-test-slave-42z9 krb5kdc[10490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:24.842784 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:25.173465 10512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:25.173777 10512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:25.173921 10512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:25.180934 10512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:25.181033 10512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:25.181104 10512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:25.181190 10512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:25.181249 10512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:25.190655 10512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44167
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:25.191847 10512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:25.193081 10512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:25.205595 10521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:25.205649 10522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:25.205653 10525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:25.206606 10512 server_base.cc:1034] running on GCE node
I20260809 13:14:25.207262 10512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:25.208487 10512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:25.209671 10512 hybrid_clock.cc:648] HybridClock initialized: now 1786281265209621 us; error 45 us; skew 500 ppm
I20260809 13:14:25.209991 10512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:25 dist-test-slave-42z9 krb5kdc[10490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:25.212932 10512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:25.214274 10512 webserver.cc:460] Webserver started at http://127.0.0.1:45189/ using document root <none> and password file <none>
I20260809 13:14:25.214874 10512 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:25.214972 10512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:25.215215 10512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:25.218091 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "53abfa86516e4edf8c948e436c03e637"
format_stamp: "Formatted at 2026-08-09 13:14:25 on dist-test-slave-42z9"
server_key: "f61a9e493917ba64e4bf488c4c7ffed3"
server_key_iv: "28d5cff36d833534a9d0803874ef622a"
server_key_version: "encryptionkey@0"
I20260809 13:14:25.218797 10512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "53abfa86516e4edf8c948e436c03e637"
format_stamp: "Formatted at 2026-08-09 13:14:25 on dist-test-slave-42z9"
server_key: "f61a9e493917ba64e4bf488c4c7ffed3"
server_key_iv: "28d5cff36d833534a9d0803874ef622a"
server_key_version: "encryptionkey@0"
I20260809 13:14:25.223605 10512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:14:25.226854 10538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:25.228065 10512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:14:25.228223 10512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "53abfa86516e4edf8c948e436c03e637"
format_stamp: "Formatted at 2026-08-09 13:14:25 on dist-test-slave-42z9"
server_key: "f61a9e493917ba64e4bf488c4c7ffed3"
server_key_iv: "28d5cff36d833534a9d0803874ef622a"
server_key_version: "encryptionkey@0"
I20260809 13:14:25.228387 10512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:25.246953 10512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:25.251189 10512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:25.251446 10512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:25.268090 10512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44167
I20260809 13:14:25.268113 10612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44167 every 8 connection(s)
I20260809 13:14:25.269399 10512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:25.273115 10613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:25.277251 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10512
I20260809 13:14:25.277477 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:25.278098 10966 external_mini_cluster.cc:1428] Setting key dc30b463133d904ece9562a66655d4f9
I20260809 13:14:25.283932 10613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637: Bootstrap starting.
I20260809 13:14:25.287550 10613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:25.288723 10613 log.cc:824] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:25.292141 10613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637: No bootstrap required, opened a new log
Aug 09 13:14:25 dist-test-slave-42z9 krb5kdc[10490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:14:25.299757 10613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } }
I20260809 13:14:25.300180 10613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:25.300289 10613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53abfa86516e4edf8c948e436c03e637, State: Initialized, Role: FOLLOWER
I20260809 13:14:25.300869 10613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } }
I20260809 13:14:25.301039 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:25.279979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54209 (local address 127.0.0.1:44167)
0809 13:14:25.280796 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:25.280808 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:25.280854 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:14:25.282600 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:25.282624 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:25.282966 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:25.283273 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:25.285094 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:25.286457 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:25.288251 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:25.288928 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:25.289054 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:25.293757 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:25.293768 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:25.293783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:25.293826 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:25.296839 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:25.297644 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:25.297650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:25.297656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:25.297812 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:25.298252 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:25.298256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:25.298258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:25.298790 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:25.299106 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:25.299593 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 13:14:25.299902 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":346,"threads_started":1}
I20260809 13:14:25.301110 10613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:25.301229 10613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:25.301482 10613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:25.304332 10613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } }
I20260809 13:14:25.304898 10613 leader_election.cc:302] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53abfa86516e4edf8c948e436c03e637; no voters:
I20260809 13:14:25.305179 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:25.305560 10613 leader_election.cc:288] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:25.305660 10623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:25.307734 10613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:25.310086 10623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [term 1 LEADER]: Becoming Leader. State: Replica: 53abfa86516e4edf8c948e436c03e637, State: Running, Role: LEADER
I20260809 13:14:25.310774 10623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } }
I20260809 13:14:25.313836 10627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53abfa86516e4edf8c948e436c03e637. Latest consensus state: current_term: 1 leader_uuid: "53abfa86516e4edf8c948e436c03e637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } } }
I20260809 13:14:25.313915 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53abfa86516e4edf8c948e436c03e637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53abfa86516e4edf8c948e436c03e637" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44167 } } }
I20260809 13:14:25.314239 10627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:25.314276 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:25.316284 10646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:25.322202 10646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:25.331677 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:25.310607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45225 (local address 127.0.0.1:44167)
0809 13:14:25.310787 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:25.310792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:25.311292 (+ 500us) server_negotiation.cc:407] Connection header received
0809 13:14:25.311462 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:25.311470 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:25.311550 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:25.311661 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:25.316881 (+ 5220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:25.318000 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:25.319794 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:25.320473 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:25.320618 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:25.323803 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:25.323812 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:25.323818 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:25.323857 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:25.327516 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:25.328470 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:25.328476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:25.328479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:25.328638 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:25.329184 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:25.329193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:25.329197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:25.329629 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:25.329882 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:25.331133 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0809 13:14:25.331312 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:14:25.340384 10646 catalog_manager.cc:1347] Generated new cluster ID: 2cae655a77ca49b8af0c8d101cfcffe4
I20260809 13:14:25.340829 10646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:25.351926 10646 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:25.353148 10646 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:25.360975 10646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53abfa86516e4edf8c948e436c03e637: Generated new TSK 0
I20260809 13:14:25.361763 10646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:25.370990 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10512
I20260809 13:14:25.372587 10605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:25.464563 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10512
W20260809 13:14:25.487584 10486 connection.cc:535] client connection to 127.0.0.1:44167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44167): Connection reset by peer (error 104)
2026-08-09T13:14:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2121 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:25 dist-test-slave-42z9 krb5kdc[10665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:25 dist-test-slave-42z9 krb5kdc[10665](info): set up 2 sockets
Aug 09 13:14:25 dist-test-slave-42z9 krb5kdc[10665](info): commencing operation
krb5kdc: starting...
W20260809 13:14:26.691427 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:26 dist-test-slave-42z9 krb5kdc[10665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:26.973487 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:27.306170 10688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:27.306435 10688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:27.306562 10688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:27.313536 10688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:27.313640 10688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:27.313715 10688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:27.313796 10688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:27.313851 10688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:27.323297 10688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:27.324472 10688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:27.325767 10688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:27.336525 10694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:27.336493 10697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:27.336552 10699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:27.337354 10688 server_base.cc:1034] running on GCE node
I20260809 13:14:27.338079 10688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:27.339282 10688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:27.340468 10688 hybrid_clock.cc:648] HybridClock initialized: now 1786281267340423 us; error 35 us; skew 500 ppm
I20260809 13:14:27.340754 10688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:27 dist-test-slave-42z9 krb5kdc[10665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:27.343659 10688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:27.344964 10688 webserver.cc:460] Webserver started at http://127.0.0.1:39279/ using document root <none> and password file <none>
I20260809 13:14:27.345592 10688 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:27.345697 10688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:27.345953 10688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:27.348753 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "9ac7e30166944fcc8f79c51572456a16"
format_stamp: "Formatted at 2026-08-09 13:14:27 on dist-test-slave-42z9"
server_key: "58269bc91a4af4694f41114eaaab1822"
server_key_iv: "1665b6ed7767ee1e2cc1d6a297ad79a3"
server_key_version: "encryptionkey@0"
I20260809 13:14:27.349476 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "9ac7e30166944fcc8f79c51572456a16"
format_stamp: "Formatted at 2026-08-09 13:14:27 on dist-test-slave-42z9"
server_key: "58269bc91a4af4694f41114eaaab1822"
server_key_iv: "1665b6ed7767ee1e2cc1d6a297ad79a3"
server_key_version: "encryptionkey@0"
I20260809 13:14:27.354211 10688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 13:14:27.357335 10712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:27.358565 10688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 13:14:27.358709 10688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "9ac7e30166944fcc8f79c51572456a16"
format_stamp: "Formatted at 2026-08-09 13:14:27 on dist-test-slave-42z9"
server_key: "58269bc91a4af4694f41114eaaab1822"
server_key_iv: "1665b6ed7767ee1e2cc1d6a297ad79a3"
server_key_version: "encryptionkey@0"
I20260809 13:14:27.358867 10688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:27.384231 10688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:27.387616 10688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:27.387854 10688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:27.404376 10688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40193
I20260809 13:14:27.404408 10787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40193 every 8 connection(s)
I20260809 13:14:27.405650 10688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:27.408771 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10688
I20260809 13:14:27.408951 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:27.409274 10788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:27.409536 10966 external_mini_cluster.cc:1428] Setting key 720cb1e33060de43656b3b6480813208
I20260809 13:14:27.422516 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:27.411169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58129 (local address 127.0.0.1:40193)
0809 13:14:27.411890 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:27.411901 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:27.411951 (+ 50us) server_negotiation.cc:407] Connection header received
0809 13:14:27.413781 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:27.413802 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:27.414161 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:27.414450 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:27.415084 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:27.416411 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:27.418152 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:27.418775 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:27.418896 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:27.420273 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:27.420288 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:27.420306 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:27.420981 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:27.421049 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:27.421074 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:27.421185 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:27.421466 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 13:14:27.421637 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":314,"threads_started":1}
I20260809 13:14:27.422618 10788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16: Bootstrap starting.
I20260809 13:14:27.425726 10788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:27.425936 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:27.426877 10788 log.cc:824] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:27.429904 10788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16: No bootstrap required, opened a new log
I20260809 13:14:27.436844 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:27.430695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42255 (local address 127.0.0.1:40193)
0809 13:14:27.430858 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:27.430863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:27.431224 (+ 361us) server_negotiation.cc:407] Connection header received
0809 13:14:27.431357 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:27.431363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:27.431443 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:27.431559 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:27.431984 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:27.433101 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:27.434377 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:27.435078 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:27.435209 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:27.436120 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:27.436126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:27.436130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:27.436154 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:27.436200 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:27.436223 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:27.436354 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:27.436465 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 13:14:27.436509 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:14:27.438879 10788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } }
I20260809 13:14:27.439198 10788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:27.439270 10788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac7e30166944fcc8f79c51572456a16, State: Initialized, Role: FOLLOWER
I20260809 13:14:27.439755 10788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } }
I20260809 13:14:27.439953 10788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:27.440049 10788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:27.440171 10788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:27.442165 10788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } }
I20260809 13:14:27.442590 10788 leader_election.cc:302] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ac7e30166944fcc8f79c51572456a16; no voters:
I20260809 13:14:27.443187 10788 leader_election.cc:288] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:27.443279 10799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:27.443997 10799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [term 1 LEADER]: Becoming Leader. State: Replica: 9ac7e30166944fcc8f79c51572456a16, State: Running, Role: LEADER
I20260809 13:14:27.444509 10799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } }
I20260809 13:14:27.445209 10788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:27.447196 10803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ac7e30166944fcc8f79c51572456a16. Latest consensus state: current_term: 1 leader_uuid: "9ac7e30166944fcc8f79c51572456a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } } }
I20260809 13:14:27.447619 10803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:27.448092 10802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ac7e30166944fcc8f79c51572456a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac7e30166944fcc8f79c51572456a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40193 } } }
I20260809 13:14:27.448386 10802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:27.449204 10812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:27.456760 10812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:27.465669 10812 catalog_manager.cc:1347] Generated new cluster ID: c1cd2c6bfd604e7e9816a074020eace3
I20260809 13:14:27.465765 10812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:27.483312 10812 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:27.484961 10812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:27.511042 10812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ac7e30166944fcc8f79c51572456a16: Generated new TSK 0
I20260809 13:14:27.512025 10812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:27.527941 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10688
W20260809 13:14:27.528972 10782 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58129
W20260809 13:14:27.529644 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10688: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:27.529809 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10688
W20260809 13:14:27.553886 10662 connection.cc:535] client connection to 127.0.0.1:40193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40193): Connection reset by peer (error 104)
2026-08-09T13:14:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:27 dist-test-slave-42z9 krb5kdc[10831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:27 dist-test-slave-42z9 krb5kdc[10831](info): set up 2 sockets
Aug 09 13:14:27 dist-test-slave-42z9 krb5kdc[10831](info): commencing operation
krb5kdc: starting...
W20260809 13:14:28.756240 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:28 dist-test-slave-42z9 krb5kdc[10831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:29.027801 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:29.358078 10848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:29.358348 10848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:29.358492 10848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:29.365298 10848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:29.365406 10848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:29.365506 10848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:29.365587 10848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:29.365648 10848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:29.374920 10848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:29.376062 10848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:29.377266 10848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:29.388065 10854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:29.388111 10858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:29.388140 10853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:29.388901 10848 server_base.cc:1034] running on GCE node
I20260809 13:14:29.389642 10848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:29.390893 10848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:29.392089 10848 hybrid_clock.cc:648] HybridClock initialized: now 1786281269392041 us; error 39 us; skew 500 ppm
I20260809 13:14:29.392400 10848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:29 dist-test-slave-42z9 krb5kdc[10831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:29.395285 10848 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:29.396646 10848 webserver.cc:460] Webserver started at http://127.0.0.1:37941/ using document root <none> and password file <none>
I20260809 13:14:29.397246 10848 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:29.397362 10848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:29.397684 10848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:29.400444 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "54e282ed074b4d03a6cb36eacf4661c2"
format_stamp: "Formatted at 2026-08-09 13:14:29 on dist-test-slave-42z9"
server_key: "fe45ab7a4f1ef7f4d987280331bddbf4"
server_key_iv: "8e796351d1e8da892e13143f5dbe48fd"
server_key_version: "encryptionkey@0"
I20260809 13:14:29.401160 10848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "54e282ed074b4d03a6cb36eacf4661c2"
format_stamp: "Formatted at 2026-08-09 13:14:29 on dist-test-slave-42z9"
server_key: "fe45ab7a4f1ef7f4d987280331bddbf4"
server_key_iv: "8e796351d1e8da892e13143f5dbe48fd"
server_key_version: "encryptionkey@0"
I20260809 13:14:29.405861 10848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:14:29.409104 10869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:29.410313 10848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:29.410493 10848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "54e282ed074b4d03a6cb36eacf4661c2"
format_stamp: "Formatted at 2026-08-09 13:14:29 on dist-test-slave-42z9"
server_key: "fe45ab7a4f1ef7f4d987280331bddbf4"
server_key_iv: "8e796351d1e8da892e13143f5dbe48fd"
server_key_version: "encryptionkey@0"
I20260809 13:14:29.410667 10848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:29.416021 10848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:29.419651 10848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:29.419909 10848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:29.436630 10848 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39539
I20260809 13:14:29.436681 10950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39539 every 8 connection(s)
I20260809 13:14:29.437871 10848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:29.441396 10951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:29.443403 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 10848
I20260809 13:14:29.443608 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:29.444128 10966 external_mini_cluster.cc:1428] Setting key d46f81506534dddef3ad02291b97f1de
I20260809 13:14:29.451284 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:29.445851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34017 (local address 127.0.0.1:39539)
0809 13:14:29.446641 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:29.446651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:29.446695 (+ 44us) server_negotiation.cc:407] Connection header received
0809 13:14:29.448239 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:29.448264 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:29.448572 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:29.448875 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:29.449880 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:29.449886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:29.449898 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:29.449913 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:29.449967 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:29.449986 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:29.450155 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:29.450256 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 13:14:29.450418 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":352,"threads_started":1}
I20260809 13:14:29.454244 10951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2: Bootstrap starting.
I20260809 13:14:29.454689 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:29.457734 10951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:29.459054 10951 log.cc:824] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:29.462143 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:29.459937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41829 (local address 127.0.0.1:39539)
0809 13:14:29.460109 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:29.460114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:29.460397 (+ 283us) server_negotiation.cc:407] Connection header received
0809 13:14:29.460582 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:29.460587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:29.460653 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:29.460748 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:29.461562 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:29.461567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:29.461571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:29.461588 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:29.461642 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:29.461662 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:29.461778 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:29.461829 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 13:14:29.461884 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 13:14:29.462339 10951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2: No bootstrap required, opened a new log
I20260809 13:14:29.469602 10951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } }
I20260809 13:14:29.469939 10951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:29.470031 10951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e282ed074b4d03a6cb36eacf4661c2, State: Initialized, Role: FOLLOWER
I20260809 13:14:29.470577 10951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } }
I20260809 13:14:29.470824 10951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:29.471043 10951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:29.471205 10951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:29.473333 10951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } }
I20260809 13:14:29.473805 10951 leader_election.cc:302] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e282ed074b4d03a6cb36eacf4661c2; no voters:
I20260809 13:14:29.474413 10951 leader_election.cc:288] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:29.474516 10972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:29.475514 10972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [term 1 LEADER]: Becoming Leader. State: Replica: 54e282ed074b4d03a6cb36eacf4661c2, State: Running, Role: LEADER
I20260809 13:14:29.475983 10972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } }
I20260809 13:14:29.476264 10951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:29.478662 10973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e282ed074b4d03a6cb36eacf4661c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } } }
I20260809 13:14:29.478705 10974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e282ed074b4d03a6cb36eacf4661c2. Latest consensus state: current_term: 1 leader_uuid: "54e282ed074b4d03a6cb36eacf4661c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e282ed074b4d03a6cb36eacf4661c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39539 } } }
I20260809 13:14:29.479094 10973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:29.479095 10974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:29.480017 10981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:29.485427 10981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:29.495293 10981 catalog_manager.cc:1347] Generated new cluster ID: 611ac04930ae470a81cacf38e2cf1751
I20260809 13:14:29.495411 10981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:29.507195 10981 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:29.507642 10981 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:29.515007 10981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e282ed074b4d03a6cb36eacf4661c2: Generated new TSK 0
I20260809 13:14:29.515827 10981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:29.520754 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10848
W20260809 13:14:29.522113 10944 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34017
W20260809 13:14:29.522926 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 10848: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:29.523083 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 10848
2026-08-09T13:14:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1994 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:29 dist-test-slave-42z9 krb5kdc[11002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:29 dist-test-slave-42z9 krb5kdc[11002](info): set up 2 sockets
Aug 09 13:14:29 dist-test-slave-42z9 krb5kdc[11002](info): commencing operation
krb5kdc: starting...
W20260809 13:14:30.730510 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:30 dist-test-slave-42z9 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:31.009668 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:31.341708 11023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:31.341964 11023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:31.342056 11023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:31.348984 11023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:31.349078 11023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:31.349129 11023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:31.349192 11023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:31.349233 11023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:31.358608 11023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:31.359764 11023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:31.360941 11023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:31.372006 11031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:31.371925 11037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:31.371973 11032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:31.372859 11023 server_base.cc:1034] running on GCE node
I20260809 13:14:31.373585 11023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:31.374876 11023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:31.376062 11023 hybrid_clock.cc:648] HybridClock initialized: now 1786281271376018 us; error 41 us; skew 500 ppm
I20260809 13:14:31.376374 11023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:31 dist-test-slave-42z9 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:31.379355 11023 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:31.380676 11023 webserver.cc:460] Webserver started at http://127.0.0.1:34173/ using document root <none> and password file <none>
I20260809 13:14:31.381275 11023 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:31.381402 11023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:31.381695 11023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:31.384459 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "9f2725c7fc46489381c12fd716abf7e4"
format_stamp: "Formatted at 2026-08-09 13:14:31 on dist-test-slave-42z9"
server_key: "2a94ac2ac71c25ddf3ebbc2d8ac89e29"
server_key_iv: "97b5010d4d844ccee3f9476440fbcff3"
server_key_version: "encryptionkey@0"
I20260809 13:14:31.385161 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "9f2725c7fc46489381c12fd716abf7e4"
format_stamp: "Formatted at 2026-08-09 13:14:31 on dist-test-slave-42z9"
server_key: "2a94ac2ac71c25ddf3ebbc2d8ac89e29"
server_key_iv: "97b5010d4d844ccee3f9476440fbcff3"
server_key_version: "encryptionkey@0"
I20260809 13:14:31.389905 11023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:14:31.393018 11045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:31.394224 11023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 13:14:31.394367 11023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "9f2725c7fc46489381c12fd716abf7e4"
format_stamp: "Formatted at 2026-08-09 13:14:31 on dist-test-slave-42z9"
server_key: "2a94ac2ac71c25ddf3ebbc2d8ac89e29"
server_key_iv: "97b5010d4d844ccee3f9476440fbcff3"
server_key_version: "encryptionkey@0"
I20260809 13:14:31.394526 11023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:31.411002 11023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:31.414911 11023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:31.415141 11023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:31.431859 11023 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36027
I20260809 13:14:31.431880 11118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36027 every 8 connection(s)
I20260809 13:14:31.433169 11023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:31.436424 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11023
I20260809 13:14:31.436582 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:31.436793 11122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:31.437045 10966 external_mini_cluster.cc:1428] Setting key 00be8600ed360ff7d9c19607a0e2b403
I20260809 13:14:31.450680 11122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4: Bootstrap starting.
Aug 09 13:14:31 dist-test-slave-42z9 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:14:31.453792 11122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:31.454874 11122 log.cc:824] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:31.457581 11122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4: No bootstrap required, opened a new log
I20260809 13:14:31.459148 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:31.438736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48927 (local address 127.0.0.1:36027)
0809 13:14:31.439599 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:31.439613 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:31.439654 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:31.441288 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:31.441311 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:31.441717 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:31.442024 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:31.443830 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:31.445040 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:31.446839 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:31.447534 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:31.447669 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:31.452240 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:31.452248 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:31.452261 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:31.452298 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:31.455234 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:31.455905 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:31.455912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:31.455918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:31.456091 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:31.456477 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:31.456481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:31.456483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:31.457059 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:31.457352 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:31.457807 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 13:14:31.458225 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":365,"threads_started":1}
I20260809 13:14:31.462893 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:31.465703 11122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260809 13:14:31.466089 11122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:31.466182 11122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2725c7fc46489381c12fd716abf7e4, State: Initialized, Role: FOLLOWER
I20260809 13:14:31.466774 11122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260809 13:14:31.467159 11122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:31.467317 11122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:31.467546 11122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:31.469749 11122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260809 13:14:31.470223 11122 leader_election.cc:302] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f2725c7fc46489381c12fd716abf7e4; no voters:
I20260809 13:14:31.470824 11122 leader_election.cc:288] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:31.470918 11140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:31.472329 11140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2725c7fc46489381c12fd716abf7e4, State: Running, Role: LEADER
I20260809 13:14:31.472981 11122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:31.472949 11140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260809 13:14:31.480746 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:31.467216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44625 (local address 127.0.0.1:36027)
0809 13:14:31.467349 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:31.467353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:31.467931 (+ 578us) server_negotiation.cc:407] Connection header received
0809 13:14:31.468108 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:31.468125 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:31.468187 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:31.468279 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:31.469714 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:31.470549 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:31.472140 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:31.472808 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:31.472922 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:31.475339 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:31.475344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:31.475348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:31.475378 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:31.478138 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:31.478761 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:31.478767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:31.478771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:31.478885 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:31.479297 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:31.479303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:31.479306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:31.479592 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:31.479832 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:31.480179 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:14:31.480381 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 13:14:31.482638 11141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f2725c7fc46489381c12fd716abf7e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260809 13:14:31.482842 11143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f2725c7fc46489381c12fd716abf7e4. Latest consensus state: current_term: 1 leader_uuid: "9f2725c7fc46489381c12fd716abf7e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2725c7fc46489381c12fd716abf7e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260809 13:14:31.483095 11141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:31.483263 11143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:31.485304 11152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:31.490492 11152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:31.498656 11152 catalog_manager.cc:1347] Generated new cluster ID: c0611cf2bf7e4bfcb108fc3255997295
I20260809 13:14:31.498783 11152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:31.513033 11152 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:31.514282 11152 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:31.524667 11152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f2725c7fc46489381c12fd716abf7e4: Generated new TSK 0
I20260809 13:14:31.525388 11152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:31.536089 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11023
I20260809 13:14:31.537042 11113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:31.625281 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11023
2026-08-09T13:14:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2100 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:31 dist-test-slave-42z9 krb5kdc[11171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:31 dist-test-slave-42z9 krb5kdc[11171](info): set up 2 sockets
Aug 09 13:14:31 dist-test-slave-42z9 krb5kdc[11171](info): commencing operation
krb5kdc: starting...
W20260809 13:14:32.861212 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:33 dist-test-slave-42z9 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:33.158993 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:33.492525 11191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:33.492784 11191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:33.492877 11191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:33.499747 11191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:33.499840 11191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:33.499891 11191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:33.499950 11191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:33.499993 11191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:33.509168 11191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:33.510303 11191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:33.511507 11191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:33.522333 11198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:33.522562 11202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:33.522581 11199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:33.523473 11191 server_base.cc:1034] running on GCE node
I20260809 13:14:33.524168 11191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:33.525477 11191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:33.526695 11191 hybrid_clock.cc:648] HybridClock initialized: now 1786281273526648 us; error 44 us; skew 500 ppm
I20260809 13:14:33.527009 11191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:33 dist-test-slave-42z9 krb5kdc[11171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:33.533218 11191 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:33.534648 11191 webserver.cc:460] Webserver started at http://127.0.0.1:45847/ using document root <none> and password file <none>
I20260809 13:14:33.535276 11191 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:33.535393 11191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:33.535677 11191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:33.538540 11191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "d3424d7fff9d43c79c9ebec894a347ef"
format_stamp: "Formatted at 2026-08-09 13:14:33 on dist-test-slave-42z9"
server_key: "0a70a3353aa01df78d76ac1d286306f9"
server_key_iv: "2558cd7ce807461b469aa9fcb8d4870a"
server_key_version: "encryptionkey@0"
I20260809 13:14:33.539292 11191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "d3424d7fff9d43c79c9ebec894a347ef"
format_stamp: "Formatted at 2026-08-09 13:14:33 on dist-test-slave-42z9"
server_key: "0a70a3353aa01df78d76ac1d286306f9"
server_key_iv: "2558cd7ce807461b469aa9fcb8d4870a"
server_key_version: "encryptionkey@0"
I20260809 13:14:33.544130 11191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:14:33.547345 11214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:33.548502 11191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 13:14:33.548671 11191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "d3424d7fff9d43c79c9ebec894a347ef"
format_stamp: "Formatted at 2026-08-09 13:14:33 on dist-test-slave-42z9"
server_key: "0a70a3353aa01df78d76ac1d286306f9"
server_key_iv: "2558cd7ce807461b469aa9fcb8d4870a"
server_key_version: "encryptionkey@0"
I20260809 13:14:33.548849 11191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:33.574534 11191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:33.578034 11191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:33.578285 11191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:33.595332 11191 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45005
I20260809 13:14:33.595355 11295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45005 every 8 connection(s)
I20260809 13:14:33.596640 11191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:33.600208 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:33.604591 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11191
I20260809 13:14:33.604751 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:33.605168 10966 external_mini_cluster.cc:1428] Setting key 205a891f108a37dda75c863702492cd3
I20260809 13:14:33.611847 11296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: Bootstrap starting.
I20260809 13:14:33.615310 11296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:33.616475 11296 log.cc:824] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:33.619418 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:33.606831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58167 (local address 127.0.0.1:45005)
0809 13:14:33.607686 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:33.607701 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:33.607758 (+ 57us) server_negotiation.cc:407] Connection header received
0809 13:14:33.610134 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:33.610162 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:33.610610 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:33.611006 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:33.611493 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:33.613428 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:33.614755 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:33.615767 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:33.615952 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:33.616950 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:33.616961 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:33.616985 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:33.617590 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:33.617663 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:33.617687 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:33.617799 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:33.617983 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 13:14:33.618237 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":370,"threads_started":1}
I20260809 13:14:33.619668 11296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: No bootstrap required, opened a new log
I20260809 13:14:33.622838 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:33.628701 11296 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } }
I20260809 13:14:33.629143 11296 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:33.629251 11296 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3424d7fff9d43c79c9ebec894a347ef, State: Initialized, Role: FOLLOWER
I20260809 13:14:33.629830 11296 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } }
I20260809 13:14:33.630057 11296 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:33.630188 11296 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:33.630371 11296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:33.633253 11296 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } }
I20260809 13:14:33.633816 11296 leader_election.cc:302] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3424d7fff9d43c79c9ebec894a347ef; no voters:
I20260809 13:14:33.634282 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:33.627725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56279 (local address 127.0.0.1:45005)
0809 13:14:33.627902 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:33.627909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:33.628393 (+ 484us) server_negotiation.cc:407] Connection header received
0809 13:14:33.628575 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:33.628579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:33.628637 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:33.628713 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:33.629317 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:33.630153 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:33.631977 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:33.632446 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:33.632535 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:33.633600 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:33.633605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:33.633608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:33.633626 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:33.633662 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:33.633676 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:33.633748 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:33.633896 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 13:14:33.633964 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:14:33.634490 11296 leader_election.cc:288] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:33.634600 11310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:33.636650 11296 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:33.637007 11310 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [term 1 LEADER]: Becoming Leader. State: Replica: d3424d7fff9d43c79c9ebec894a347ef, State: Running, Role: LEADER
I20260809 13:14:33.637595 11310 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } }
I20260809 13:14:33.642405 11313 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3424d7fff9d43c79c9ebec894a347ef. Latest consensus state: current_term: 1 leader_uuid: "d3424d7fff9d43c79c9ebec894a347ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } } }
I20260809 13:14:33.642771 11313 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:33.643265 11312 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3424d7fff9d43c79c9ebec894a347ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3424d7fff9d43c79c9ebec894a347ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45005 } } }
I20260809 13:14:33.643569 11312 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef [sys.catalog]: This master's current role is: LEADER
W20260809 13:14:33.646301 11328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:14:33.646432 11328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:14:33.646596 11329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:33.648276 11329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:33.655646 11329 catalog_manager.cc:1347] Generated new cluster ID: c7af6e9fb6234414919292fe775b38fa
I20260809 13:14:33.655750 11329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:33.662194 11329 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:33.663281 11329 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:33.670462 11329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3424d7fff9d43c79c9ebec894a347ef: Generated new TSK 0
I20260809 13:14:33.671156 11329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:33.682456 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11191
W20260809 13:14:33.683758 11289 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58167
W20260809 13:14:33.684412 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:33.684531 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11191
2026-08-09T13:14:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2061 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:33 dist-test-slave-42z9 krb5kdc[11337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:33 dist-test-slave-42z9 krb5kdc[11337](info): set up 2 sockets
Aug 09 13:14:33 dist-test-slave-42z9 krb5kdc[11337](info): commencing operation
krb5kdc: starting...
W20260809 13:14:34.915112 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:35 dist-test-slave-42z9 krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:35.229611 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36253
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:35.572814 11354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:35.573108 11354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:35.573251 11354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:35.580322 11354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:35.580437 11354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:35.580518 11354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:35.580605 11354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:35.580667 11354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:35.590273 11354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:35.591461 11354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:35.592959 11354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:35.604094 11369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:35.604069 11366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:35.604561 11365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:35.604818 11354 server_base.cc:1034] running on GCE node
I20260809 13:14:35.605660 11354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:35.606974 11354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:35.608191 11354 hybrid_clock.cc:648] HybridClock initialized: now 1786281275608137 us; error 45 us; skew 500 ppm
I20260809 13:14:35.608513 11354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:35 dist-test-slave-42z9 krb5kdc[11337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:35.611583 11354 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:35.612939 11354 webserver.cc:460] Webserver started at http://127.0.0.1:33077/ using document root <none> and password file <none>
I20260809 13:14:35.613725 11354 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:35.613842 11354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:35.614141 11354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:35.617044 11354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "478c5dd83d79438e9ef7ff2189bb083f"
format_stamp: "Formatted at 2026-08-09 13:14:35 on dist-test-slave-42z9"
server_key: "e3c549afd6aeded76e2516362d43a3af"
server_key_iv: "7602cfc4bd2a95783022627f00417f79"
server_key_version: "encryptionkey@0"
I20260809 13:14:35.617835 11354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "478c5dd83d79438e9ef7ff2189bb083f"
format_stamp: "Formatted at 2026-08-09 13:14:35 on dist-test-slave-42z9"
server_key: "e3c549afd6aeded76e2516362d43a3af"
server_key_iv: "7602cfc4bd2a95783022627f00417f79"
server_key_version: "encryptionkey@0"
I20260809 13:14:35.622998 11354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:14:35.626400 11379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:35.627718 11354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 13:14:35.627885 11354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "478c5dd83d79438e9ef7ff2189bb083f"
format_stamp: "Formatted at 2026-08-09 13:14:35 on dist-test-slave-42z9"
server_key: "e3c549afd6aeded76e2516362d43a3af"
server_key_iv: "7602cfc4bd2a95783022627f00417f79"
server_key_version: "encryptionkey@0"
I20260809 13:14:35.628067 11354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:35.633723 11354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:35.637364 11354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:35.637657 11354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:35.653797 11354 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36253
I20260809 13:14:35.653841 11458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36253 every 8 connection(s)
I20260809 13:14:35.655064 11354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:35.658640 11459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:35.661015 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11354
I20260809 13:14:35.661178 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:35.661638 10966 external_mini_cluster.cc:1428] Setting key c9ef6385fc84f4fd440f3c1c07698985
I20260809 13:14:35.668999 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:35.663288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32911 (local address 127.0.0.1:36253)
0809 13:14:35.664060 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:35.664070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:35.664112 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:14:35.665778 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:35.665800 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:35.666141 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:35.666466 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:35.667533 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:35.667539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:35.667552 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:35.667569 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:35.667633 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:35.667656 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:35.667819 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:35.667922 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 13:14:35.668105 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":347,"threads_started":1}
I20260809 13:14:35.669992 11459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f: Bootstrap starting.
I20260809 13:14:35.672534 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:35.673272 11459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:35.674531 11459 log.cc:824] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:35.677261 11459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f: No bootstrap required, opened a new log
I20260809 13:14:35.679792 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:35.677539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60057 (local address 127.0.0.1:36253)
0809 13:14:35.677705 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:35.677710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:35.678049 (+ 339us) server_negotiation.cc:407] Connection header received
0809 13:14:35.678180 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:35.678185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:35.678238 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:35.678336 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:35.679155 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:35.679161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:35.679166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:35.679185 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:35.679239 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:35.679258 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:35.679373 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:35.679443 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 13:14:35.679501 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:14:35.685806 11459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } }
I20260809 13:14:35.686153 11459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:35.686228 11459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478c5dd83d79438e9ef7ff2189bb083f, State: Initialized, Role: FOLLOWER
I20260809 13:14:35.686702 11459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } }
I20260809 13:14:35.686901 11459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:35.687000 11459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:35.687129 11459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:35.689263 11459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } }
I20260809 13:14:35.689785 11459 leader_election.cc:302] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 478c5dd83d79438e9ef7ff2189bb083f; no voters:
I20260809 13:14:35.690284 11459 leader_election.cc:288] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:35.690407 11473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:35.692301 11459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:35.693712 11473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [term 1 LEADER]: Becoming Leader. State: Replica: 478c5dd83d79438e9ef7ff2189bb083f, State: Running, Role: LEADER
I20260809 13:14:35.694169 11473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } }
I20260809 13:14:35.696919 11474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "478c5dd83d79438e9ef7ff2189bb083f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } } }
I20260809 13:14:35.696973 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 478c5dd83d79438e9ef7ff2189bb083f. Latest consensus state: current_term: 1 leader_uuid: "478c5dd83d79438e9ef7ff2189bb083f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "478c5dd83d79438e9ef7ff2189bb083f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36253 } } }
I20260809 13:14:35.697371 11474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:35.697368 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:35.699384 11488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:35.703774 11488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:35.712133 11488 catalog_manager.cc:1347] Generated new cluster ID: 6f3c56a992f447c6bbfece65f94bbe73
I20260809 13:14:35.712250 11488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:35.720721 11488 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:35.721095 11488 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:35.726935 11488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 478c5dd83d79438e9ef7ff2189bb083f: Generated new TSK 0
I20260809 13:14:35.727718 11488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:35.733208 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11354
W20260809 13:14:35.734506 11451 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32911
W20260809 13:14:35.735329 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11354: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:35.735489 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11354
2026-08-09T13:14:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:35 dist-test-slave-42z9 krb5kdc[11504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:35 dist-test-slave-42z9 krb5kdc[11504](info): set up 2 sockets
Aug 09 13:14:35 dist-test-slave-42z9 krb5kdc[11504](info): commencing operation
krb5kdc: starting...
W20260809 13:14:36.965689 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:37 dist-test-slave-42z9 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:37.267756 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:37.600956 11527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:37.601210 11527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:37.601307 11527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:37.608412 11527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:37.608515 11527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:37.608567 11527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:37.608628 11527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:37.608670 11527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:37.618206 11527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:37.619338 11527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:37.620606 11527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:37.631578 11535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:37.631569 11538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:37.631548 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:37.632305 11527 server_base.cc:1034] running on GCE node
I20260809 13:14:37.632997 11527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:37.634316 11527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:37.635517 11527 hybrid_clock.cc:648] HybridClock initialized: now 1786281277635471 us; error 26 us; skew 500 ppm
I20260809 13:14:37.635833 11527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:37 dist-test-slave-42z9 krb5kdc[11504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:37.638829 11527 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:37.640250 11527 webserver.cc:460] Webserver started at http://127.0.0.1:46183/ using document root <none> and password file <none>
I20260809 13:14:37.640900 11527 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:37.641019 11527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:37.641302 11527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:37.644126 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "3d989c29e17f4e0e9e648142aeb03257"
format_stamp: "Formatted at 2026-08-09 13:14:37 on dist-test-slave-42z9"
server_key: "8f860e34231cbe288030d9df7ed7114f"
server_key_iv: "9329370a36ab3010cc878a7d36ad7b0e"
server_key_version: "encryptionkey@0"
I20260809 13:14:37.644860 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "3d989c29e17f4e0e9e648142aeb03257"
format_stamp: "Formatted at 2026-08-09 13:14:37 on dist-test-slave-42z9"
server_key: "8f860e34231cbe288030d9df7ed7114f"
server_key_iv: "9329370a36ab3010cc878a7d36ad7b0e"
server_key_version: "encryptionkey@0"
I20260809 13:14:37.649612 11527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260809 13:14:37.652783 11548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:37.654013 11527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:37.654175 11527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "3d989c29e17f4e0e9e648142aeb03257"
format_stamp: "Formatted at 2026-08-09 13:14:37 on dist-test-slave-42z9"
server_key: "8f860e34231cbe288030d9df7ed7114f"
server_key_iv: "9329370a36ab3010cc878a7d36ad7b0e"
server_key_version: "encryptionkey@0"
I20260809 13:14:37.654342 11527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:37.681835 11527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:37.685935 11527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:37.686197 11527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:37.703744 11527 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39123
I20260809 13:14:37.703776 11631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39123 every 8 connection(s)
I20260809 13:14:37.705024 11527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:37.708592 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:37.714939 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11527
I20260809 13:14:37.715135 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:37.715687 10966 external_mini_cluster.cc:1428] Setting key a5ac241e09369402aa1af3f554fd3b65
I20260809 13:14:37.719769 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: Bootstrap starting.
I20260809 13:14:37.722769 11632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:37.723803 11632 log.cc:824] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:37.726331 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: No bootstrap required, opened a new log
Aug 09 13:14:37 dist-test-slave-42z9 krb5kdc[11504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:14:37.734522 11632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } }
I20260809 13:14:37.734885 11632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:37.735002 11632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d989c29e17f4e0e9e648142aeb03257, State: Initialized, Role: FOLLOWER
I20260809 13:14:37.735554 11632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } }
I20260809 13:14:37.735792 11632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:37.735930 11632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:37.736152 11632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:37.737936 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:37.717674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:59385 (local address 127.0.0.1:39123)
0809 13:14:37.718899 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:37.718911 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:37.718952 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:37.720632 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:37.720655 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:37.721031 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:37.721390 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:37.722821 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:37.724163 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:37.725414 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:37.726092 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:37.726269 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:37.730666 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:37.730675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:37.730690 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:37.730727 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:37.733715 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:37.734562 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:37.734568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:37.734573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:37.734724 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:37.735213 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:37.735217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:37.735219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:37.735777 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:37.736093 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:37.736574 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 13:14:37.736886 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":792,"threads_started":1}
I20260809 13:14:37.738628 11632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } }
I20260809 13:14:37.739094 11632 leader_election.cc:302] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d989c29e17f4e0e9e648142aeb03257; no voters:
I20260809 13:14:37.739665 11632 leader_election.cc:288] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:37.739806 11640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:37.740953 11640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [term 1 LEADER]: Becoming Leader. State: Replica: 3d989c29e17f4e0e9e648142aeb03257, State: Running, Role: LEADER
I20260809 13:14:37.741569 11640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } }
I20260809 13:14:37.741864 11632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:37.742153 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:37.747594 11646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d989c29e17f4e0e9e648142aeb03257. Latest consensus state: current_term: 1 leader_uuid: "3d989c29e17f4e0e9e648142aeb03257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } } }
I20260809 13:14:37.748034 11646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:37.747561 11644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d989c29e17f4e0e9e648142aeb03257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d989c29e17f4e0e9e648142aeb03257" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39123 } } }
I20260809 13:14:37.748397 11644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257 [sys.catalog]: This master's current role is: LEADER
W20260809 13:14:37.758096 11669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:14:37.758247 11669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:14:37.758417 11676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:37.760659 11676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:37.769865 11676 catalog_manager.cc:1347] Generated new cluster ID: 24130510cf574e79abc0e560c6ddd44c
I20260809 13:14:37.770030 11676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:37.771185 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:37.755527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:49919 (local address 127.0.0.1:39123)
0809 13:14:37.755688 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:37.755694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:37.756233 (+ 539us) server_negotiation.cc:407] Connection header received
0809 13:14:37.756360 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:37.756365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:37.756440 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:37.756540 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:37.758351 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:37.759548 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:37.761292 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:37.761964 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:37.762107 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:37.764835 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:37.764841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:37.764844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:37.764874 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:37.767883 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:37.768759 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:37.768764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:37.768768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:37.768948 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:37.769482 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:37.769487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:37.769490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:37.769919 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:37.770206 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:37.770573 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 13:14:37.770742 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:14:37.782820 11676 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:37.783980 11676 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:37.793294 11676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d989c29e17f4e0e9e648142aeb03257: Generated new TSK 0
I20260809 13:14:37.794129 11676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:37.799062 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11527
I20260809 13:14:37.800669 11618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:37.888132 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11527
W20260809 13:14:37.911517 11502 connection.cc:535] client connection to 127.0.0.1:39123 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39123): Connection reset by peer (error 104)
2026-08-09T13:14:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:38 dist-test-slave-42z9 krb5kdc[11690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:38 dist-test-slave-42z9 krb5kdc[11690](info): set up 2 sockets
Aug 09 13:14:38 dist-test-slave-42z9 krb5kdc[11690](info): commencing operation
krb5kdc: starting...
W20260809 13:14:39.109615 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:39 dist-test-slave-42z9 krb5kdc[11690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:39.398263 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:39.723896 11710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:39.724174 11710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:39.724314 11710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:39.731312 11710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:39.731413 11710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:39.731488 11710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:39.731580 11710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:39.731643 11710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:39.741096 11710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:39.742311 11710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:39.743546 11710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:39.754449 11716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:39.754468 11717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:39.754712 11719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:39.755159 11710 server_base.cc:1034] running on GCE node
I20260809 13:14:39.755864 11710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:39.757102 11710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:39.758303 11710 hybrid_clock.cc:648] HybridClock initialized: now 1786281279758266 us; error 28 us; skew 500 ppm
I20260809 13:14:39.758639 11710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:39 dist-test-slave-42z9 krb5kdc[11690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:39.764735 11710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:39.766175 11710 webserver.cc:460] Webserver started at http://127.0.0.1:45069/ using document root <none> and password file <none>
I20260809 13:14:39.766819 11710 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:39.766939 11710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:39.767238 11710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:39.770167 11710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "b61686c0361844628ceb418d4bd3e467"
format_stamp: "Formatted at 2026-08-09 13:14:39 on dist-test-slave-42z9"
server_key: "48bd7e8ca174475c8491823880bdfc77"
server_key_iv: "996f99f8c2022d02296b900c0950ca1e"
server_key_version: "encryptionkey@0"
I20260809 13:14:39.770915 11710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "b61686c0361844628ceb418d4bd3e467"
format_stamp: "Formatted at 2026-08-09 13:14:39 on dist-test-slave-42z9"
server_key: "48bd7e8ca174475c8491823880bdfc77"
server_key_iv: "996f99f8c2022d02296b900c0950ca1e"
server_key_version: "encryptionkey@0"
I20260809 13:14:39.775768 11710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 13:14:39.779117 11730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:39.780355 11710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 13:14:39.780515 11710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "b61686c0361844628ceb418d4bd3e467"
format_stamp: "Formatted at 2026-08-09 13:14:39 on dist-test-slave-42z9"
server_key: "48bd7e8ca174475c8491823880bdfc77"
server_key_iv: "996f99f8c2022d02296b900c0950ca1e"
server_key_version: "encryptionkey@0"
I20260809 13:14:39.780686 11710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:39.793931 11710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:39.798110 11710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:39.798372 11710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:39.815245 11710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32897
I20260809 13:14:39.815277 11809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32897 every 8 connection(s)
I20260809 13:14:39.816587 11710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:39.820188 11810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:39.821570 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11710
I20260809 13:14:39.821777 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:39.822336 10966 external_mini_cluster.cc:1428] Setting key 629754a68b5e6d76aebba812aa97d65d
I20260809 13:14:39.832461 11810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467: Bootstrap starting.
I20260809 13:14:39.834851 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:39.824197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:49159 (local address 127.0.0.1:32897)
0809 13:14:39.824936 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:39.824947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:39.824990 (+ 43us) server_negotiation.cc:407] Connection header received
0809 13:14:39.826628 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:39.826651 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:39.827026 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:39.827393 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:39.827971 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:39.829265 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:39.830630 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:39.831279 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:39.831445 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:39.832690 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:39.832698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:39.832712 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:39.833225 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:39.833299 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:39.833327 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:39.833507 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:39.833804 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 13:14:39.833954 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":309,"threads_started":1}
I20260809 13:14:39.835704 11810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:39.836808 11810 log.cc:824] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:39.838639 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:39.839552 11810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467: No bootstrap required, opened a new log
I20260809 13:14:39.848141 11810 raft_consensus.cc:348] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260809 13:14:39.848577 11810 raft_consensus.cc:374] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:39.848708 11810 raft_consensus.cc:729] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61686c0361844628ceb418d4bd3e467, State: Initialized, Role: FOLLOWER
I20260809 13:14:39.849354 11810 consensus_queue.cc:260] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260809 13:14:39.849653 11810 raft_consensus.cc:388] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:39.849808 11810 raft_consensus.cc:482] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:39.850039 11810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:39.850149 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:39.843584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:54475 (local address 127.0.0.1:32897)
0809 13:14:39.843763 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:39.843768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:39.844039 (+ 271us) server_negotiation.cc:407] Connection header received
0809 13:14:39.844221 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:39.844228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:39.844288 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:39.844383 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:39.844903 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:39.845777 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:39.847568 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:39.847983 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:39.848073 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:39.849294 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:39.849299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:39.849302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:39.849319 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:39.849361 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:39.849375 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:39.849489 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:39.849789 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 13:14:39.849844 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 13:14:39.852473 11810 raft_consensus.cc:504] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260809 13:14:39.852916 11810 leader_election.cc:302] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b61686c0361844628ceb418d4bd3e467; no voters:
I20260809 13:14:39.853413 11810 leader_election.cc:288] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:39.853569 11821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:39.854946 11821 raft_consensus.cc:686] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [term 1 LEADER]: Becoming Leader. State: Replica: b61686c0361844628ceb418d4bd3e467, State: Running, Role: LEADER
I20260809 13:14:39.855532 11810 sys_catalog.cc:564] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:39.855477 11821 consensus_queue.cc:237] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260809 13:14:39.858018 11822 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b61686c0361844628ceb418d4bd3e467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260809 13:14:39.857986 11823 sys_catalog.cc:455] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b61686c0361844628ceb418d4bd3e467. Latest consensus state: current_term: 1 leader_uuid: "b61686c0361844628ceb418d4bd3e467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b61686c0361844628ceb418d4bd3e467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260809 13:14:39.858398 11823 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:39.858407 11822 sys_catalog.cc:458] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:39.860692 11828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:39.865675 11828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:39.875026 11828 catalog_manager.cc:1347] Generated new cluster ID: 796f45f86afa4261ba34ef2c0c614654
I20260809 13:14:39.875144 11828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:39.895150 11828 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:39.896402 11828 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:39.901566 11828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b61686c0361844628ceb418d4bd3e467: Generated new TSK 0
I20260809 13:14:39.902376 11828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:39.916551 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11710
W20260809 13:14:39.918044 11803 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.33:49159
W20260809 13:14:39.919055 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11710: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:39.919185 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11710
W20260809 13:14:39.944201 11687 connection.cc:535] client connection to 127.0.0.1:32897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32897): Connection reset by peer (error 104)
2026-08-09T13:14:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2032 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:40 dist-test-slave-42z9 krb5kdc[11847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:40 dist-test-slave-42z9 krb5kdc[11847](info): set up 2 sockets
Aug 09 13:14:40 dist-test-slave-42z9 krb5kdc[11847](info): commencing operation
krb5kdc: starting...
W20260809 13:14:41.148576 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:41 dist-test-slave-42z9 krb5kdc[11847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:41.441991 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:41.768191 11869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:41.768453 11869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:41.768587 11869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:41.775489 11869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:41.775588 11869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:41.775662 11869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:41.775738 11869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:41.775797 11869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:41.785115 11869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:41.786306 11869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:41.787535 11869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:41.801826 11876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:41.801868 11878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:41.801791 11880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:41.802708 11869 server_base.cc:1034] running on GCE node
I20260809 13:14:41.803644 11869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:41.805459 11869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:41.806691 11869 hybrid_clock.cc:648] HybridClock initialized: now 1786281281806631 us; error 57 us; skew 500 ppm
I20260809 13:14:41.807019 11869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:41 dist-test-slave-42z9 krb5kdc[11847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:41.812139 11869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:41.813886 11869 webserver.cc:460] Webserver started at http://127.0.0.1:41209/ using document root <none> and password file <none>
I20260809 13:14:41.814603 11869 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:41.814731 11869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:41.815032 11869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:41.818104 11869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "aebc3c31a0144dd8b00aae1f4de642a6"
format_stamp: "Formatted at 2026-08-09 13:14:41 on dist-test-slave-42z9"
server_key: "cc9da933ab314b99d51fdaaf0e4fbe73"
server_key_iv: "d3785ca1c58996e4910cf85ff55dc6eb"
server_key_version: "encryptionkey@0"
I20260809 13:14:41.818976 11869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "aebc3c31a0144dd8b00aae1f4de642a6"
format_stamp: "Formatted at 2026-08-09 13:14:41 on dist-test-slave-42z9"
server_key: "cc9da933ab314b99d51fdaaf0e4fbe73"
server_key_iv: "d3785ca1c58996e4910cf85ff55dc6eb"
server_key_version: "encryptionkey@0"
I20260809 13:14:41.826468 11869 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260809 13:14:41.830921 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:41.832410 11869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:41.832580 11869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "aebc3c31a0144dd8b00aae1f4de642a6"
format_stamp: "Formatted at 2026-08-09 13:14:41 on dist-test-slave-42z9"
server_key: "cc9da933ab314b99d51fdaaf0e4fbe73"
server_key_iv: "d3785ca1c58996e4910cf85ff55dc6eb"
server_key_version: "encryptionkey@0"
I20260809 13:14:41.832772 11869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:41.839982 11869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:41.844282 11869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:41.844519 11869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:41.860987 11869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39077
I20260809 13:14:41.861052 11969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39077 every 8 connection(s)
I20260809 13:14:41.862360 11869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:41.865929 11973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:41.870834 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 11869
I20260809 13:14:41.871029 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:41.871544 10966 external_mini_cluster.cc:1428] Setting key e6b78319811b61b3ff35f08524659459
I20260809 13:14:41.876828 11973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6: Bootstrap starting.
I20260809 13:14:41.879191 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:41.873371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:53481 (local address 127.0.0.1:39077)
0809 13:14:41.874141 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:41.874153 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:41.874200 (+ 47us) server_negotiation.cc:407] Connection header received
0809 13:14:41.875881 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:41.875903 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:41.876230 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:41.876557 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:41.877663 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:41.877670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:41.877682 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:41.877706 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:41.877762 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:41.877781 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:41.878012 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:41.878207 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 13:14:41.878368 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":298,"threads_started":1}
I20260809 13:14:41.880010 11973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:41.881102 11973 log.cc:824] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:41.882552 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:41.883661 11973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6: No bootstrap required, opened a new log
I20260809 13:14:41.888995 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:41.886799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:60613 (local address 127.0.0.1:39077)
0809 13:14:41.887363 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:41.887373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:41.887392 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:14:41.887442 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:41.887447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:41.887501 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:41.887612 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:41.888398 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:41.888402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:41.888407 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:41.888429 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:41.888473 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:41.888494 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:41.888616 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:41.888665 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 13:14:41.888717 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449}
I20260809 13:14:41.891791 11973 raft_consensus.cc:348] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260809 13:14:41.892163 11973 raft_consensus.cc:374] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:41.892279 11973 raft_consensus.cc:729] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebc3c31a0144dd8b00aae1f4de642a6, State: Initialized, Role: FOLLOWER
I20260809 13:14:41.892795 11973 consensus_queue.cc:260] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260809 13:14:41.893003 11973 raft_consensus.cc:388] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:41.893131 11973 raft_consensus.cc:482] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:41.893282 11973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:41.895416 11973 raft_consensus.cc:504] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260809 13:14:41.895857 11973 leader_election.cc:302] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aebc3c31a0144dd8b00aae1f4de642a6; no voters:
I20260809 13:14:41.896376 11973 leader_election.cc:288] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:41.896492 11985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:41.897099 11985 raft_consensus.cc:686] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [term 1 LEADER]: Becoming Leader. State: Replica: aebc3c31a0144dd8b00aae1f4de642a6, State: Running, Role: LEADER
I20260809 13:14:41.897728 11985 consensus_queue.cc:237] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260809 13:14:41.898265 11973 sys_catalog.cc:564] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:41.900769 11988 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aebc3c31a0144dd8b00aae1f4de642a6. Latest consensus state: current_term: 1 leader_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } } }
I20260809 13:14:41.901190 11988 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:41.901868 11986 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc3c31a0144dd8b00aae1f4de642a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } } }
I20260809 13:14:41.902210 11986 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:41.906702 12000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:41.911149 12000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:41.919109 12000 catalog_manager.cc:1347] Generated new cluster ID: 54e58af4385d430691b242a1b9bcf1c1
I20260809 13:14:41.919210 12000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:41.930464 12000 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:41.930862 12000 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:41.940774 12000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aebc3c31a0144dd8b00aae1f4de642a6: Generated new TSK 0
I20260809 13:14:41.941717 12000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:41.957160 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11869
W20260809 13:14:41.958666 11964 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.33:53481
W20260809 13:14:41.959250 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 11869: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:41.959370 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 11869
2026-08-09T13:14:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2042 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:42 dist-test-slave-42z9 krb5kdc[12016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:42 dist-test-slave-42z9 krb5kdc[12016](info): set up 2 sockets
Aug 09 13:14:42 dist-test-slave-42z9 krb5kdc[12016](info): commencing operation
krb5kdc: starting...
W20260809 13:14:43.194108 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:43 dist-test-slave-42z9 krb5kdc[12016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:43.486183 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:43.814565 12033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:43.814868 12033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:43.815016 12033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:43.822038 12033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:43.822151 12033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:43.822229 12033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:43.822311 12033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:43.822371 12033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:43.831715 12033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:43.832892 12033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:43.834169 12033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:43.845351 12043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:43.845346 12048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:43.845402 12046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:43.846732 12033 server_base.cc:1034] running on GCE node
I20260809 13:14:43.847553 12033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:43.848920 12033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:43.850131 12033 hybrid_clock.cc:648] HybridClock initialized: now 1786281283850060 us; error 46 us; skew 500 ppm
I20260809 13:14:43.850471 12033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:43 dist-test-slave-42z9 krb5kdc[12016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:43.853415 12033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:43.854797 12033 webserver.cc:460] Webserver started at http://127.0.0.1:46479/ using document root <none> and password file <none>
I20260809 13:14:43.855463 12033 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:43.855584 12033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:43.855880 12033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:43.858883 12033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "d48a456ac91c4f9196c8288c7f5dec4a"
format_stamp: "Formatted at 2026-08-09 13:14:43 on dist-test-slave-42z9"
server_key: "c6601ddebe55ab31eed8799a136fb775"
server_key_iv: "dffd09fea669e5a228c7b3b89b0ce478"
server_key_version: "encryptionkey@0"
I20260809 13:14:43.859652 12033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "d48a456ac91c4f9196c8288c7f5dec4a"
format_stamp: "Formatted at 2026-08-09 13:14:43 on dist-test-slave-42z9"
server_key: "c6601ddebe55ab31eed8799a136fb775"
server_key_iv: "dffd09fea669e5a228c7b3b89b0ce478"
server_key_version: "encryptionkey@0"
I20260809 13:14:43.864627 12033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 13:14:43.867991 12059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:43.869230 12033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:14:43.869397 12033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "d48a456ac91c4f9196c8288c7f5dec4a"
format_stamp: "Formatted at 2026-08-09 13:14:43 on dist-test-slave-42z9"
server_key: "c6601ddebe55ab31eed8799a136fb775"
server_key_iv: "dffd09fea669e5a228c7b3b89b0ce478"
server_key_version: "encryptionkey@0"
I20260809 13:14:43.869642 12033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:43.880044 12033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:43.883735 12033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:43.883981 12033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:43.901149 12033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39201
I20260809 13:14:43.901183 12136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39201 every 8 connection(s)
I20260809 13:14:43.902508 12033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:43.906064 12137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:43.909426 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12033
I20260809 13:14:43.909654 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:43.910079 10966 external_mini_cluster.cc:1428] Setting key ec4a37f4947f811bc4f253b039459d5f
I20260809 13:14:43.917629 12137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a: Bootstrap starting.
I20260809 13:14:43.921239 12137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:43.922290 12137 log.cc:824] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:43.924840 12137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a: No bootstrap required, opened a new log
Aug 09 13:14:43 dist-test-slave-42z9 krb5kdc[12016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 13:14:43.932633 12137 raft_consensus.cc:348] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260809 13:14:43.933030 12137 raft_consensus.cc:374] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:43.933156 12137 raft_consensus.cc:729] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48a456ac91c4f9196c8288c7f5dec4a, State: Initialized, Role: FOLLOWER
I20260809 13:14:43.933748 12137 consensus_queue.cc:260] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260809 13:14:43.933862 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:43.911838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:60355 (local address 127.0.0.1:39201)
0809 13:14:43.912667 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:43.912682 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:43.913063 (+ 381us) server_negotiation.cc:407] Connection header received
0809 13:14:43.915895 (+ 2832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:43.915924 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:43.916390 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:43.916824 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:43.918222 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:43.920210 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:43.921478 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:43.922175 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:43.922350 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:43.926460 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:43.926468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:43.926482 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:43.926528 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:43.929635 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:43.930400 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:43.930407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:43.930413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:43.930605 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:43.931069 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:43.931073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:43.931075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:43.931591 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:43.931917 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:43.932410 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 13:14:43.932735 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":344,"threads_started":1}
I20260809 13:14:43.934000 12137 raft_consensus.cc:388] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:43.934185 12137 raft_consensus.cc:482] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:43.934391 12137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:43.937047 12137 raft_consensus.cc:504] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260809 13:14:43.937582 12137 leader_election.cc:302] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d48a456ac91c4f9196c8288c7f5dec4a; no voters:
I20260809 13:14:43.937700 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:43.938134 12137 leader_election.cc:288] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:43.938262 12144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:43.938865 12144 raft_consensus.cc:686] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [term 1 LEADER]: Becoming Leader. State: Replica: d48a456ac91c4f9196c8288c7f5dec4a, State: Running, Role: LEADER
I20260809 13:14:43.939457 12144 consensus_queue.cc:237] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } }
I20260809 13:14:43.943074 12147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } } }
I20260809 13:14:43.943485 12147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:43.948176 12147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d48a456ac91c4f9196c8288c7f5dec4a. Latest consensus state: current_term: 1 leader_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d48a456ac91c4f9196c8288c7f5dec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39201 } } }
I20260809 13:14:43.948432 12147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:43.954066 12137 sys_catalog.cc:564] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:43.954258 12154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:43.961496 12154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:43.968968 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:43.952872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:57421 (local address 127.0.0.1:39201)
0809 13:14:43.953028 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:43.953034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:43.953054 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:14:43.953215 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:43.953224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:43.953292 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:43.953372 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:43.954878 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:43.955952 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:43.957244 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:43.957826 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:43.957958 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:43.961170 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:43.961177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:43.961182 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:43.961227 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:43.964958 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:43.965993 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:43.965998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:43.966002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:43.966143 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:43.966762 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:43.966767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:43.966771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:43.967214 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:43.968412 (+ 1198us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:43.968593 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 13:14:43.968726 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:14:43.977259 12154 catalog_manager.cc:1347] Generated new cluster ID: 698a796b31e1431a9cf2b42b75f7dc7b
I20260809 13:14:43.977377 12154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:43.996217 12154 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:43.997663 12154 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:44.011556 12154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d48a456ac91c4f9196c8288c7f5dec4a: Generated new TSK 0
I20260809 13:14:44.012650 12154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:44.026896 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12033
I20260809 13:14:44.027938 12128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:44.120362 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12033
W20260809 13:14:44.145119 12013 connection.cc:535] client connection to 127.0.0.1:39201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39201): Connection reset by peer (error 104)
2026-08-09T13:14:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:44 dist-test-slave-42z9 krb5kdc[12195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:44 dist-test-slave-42z9 krb5kdc[12195](info): set up 2 sockets
Aug 09 13:14:44 dist-test-slave-42z9 krb5kdc[12195](info): commencing operation
krb5kdc: starting...
W20260809 13:14:45.356560 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:45 dist-test-slave-42z9 krb5kdc[12195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:45.659969 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33885
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:45.985673 12212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:45.985946 12212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:45.986079 12212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:45.993149 12212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:45.993257 12212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:45.993335 12212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:45.993429 12212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:45.993500 12212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:46.003216 12212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:46.004464 12212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:46.005815 12212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:46.017256 12220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:46.017277 12226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:46.017220 12219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:46.017999 12212 server_base.cc:1034] running on GCE node
I20260809 13:14:46.018761 12212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:46.020169 12212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:46.021375 12212 hybrid_clock.cc:648] HybridClock initialized: now 1786281286021323 us; error 45 us; skew 500 ppm
I20260809 13:14:46.021732 12212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:46 dist-test-slave-42z9 krb5kdc[12195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:46.027824 12212 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:46.029204 12212 webserver.cc:460] Webserver started at http://127.0.0.1:34887/ using document root <none> and password file <none>
I20260809 13:14:46.029911 12212 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:46.030038 12212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:46.030332 12212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:46.033308 12212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "c30cda7d99784457ace4da5685a87302"
format_stamp: "Formatted at 2026-08-09 13:14:46 on dist-test-slave-42z9"
server_key: "e883be560a81898a559d6a2d164d68bf"
server_key_iv: "3bd09a383c60ef46ede4ddbdeb983d1c"
server_key_version: "encryptionkey@0"
I20260809 13:14:46.034107 12212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "c30cda7d99784457ace4da5685a87302"
format_stamp: "Formatted at 2026-08-09 13:14:46 on dist-test-slave-42z9"
server_key: "e883be560a81898a559d6a2d164d68bf"
server_key_iv: "3bd09a383c60ef46ede4ddbdeb983d1c"
server_key_version: "encryptionkey@0"
I20260809 13:14:46.039151 12212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 13:14:46.042614 12237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:46.043840 12212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:46.043999 12212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "c30cda7d99784457ace4da5685a87302"
format_stamp: "Formatted at 2026-08-09 13:14:46 on dist-test-slave-42z9"
server_key: "e883be560a81898a559d6a2d164d68bf"
server_key_iv: "3bd09a383c60ef46ede4ddbdeb983d1c"
server_key_version: "encryptionkey@0"
I20260809 13:14:46.044183 12212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:46.064996 12212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:46.068962 12212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:46.069212 12212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:46.086109 12212 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33885
I20260809 13:14:46.086169 12320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33885 every 8 connection(s)
I20260809 13:14:46.087404 12212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:46.091045 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:46.093618 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12212
I20260809 13:14:46.093786 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:46.094342 10966 external_mini_cluster.cc:1428] Setting key c2a9947c20aba3a07fb740073c674295
I20260809 13:14:46.102748 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302: Bootstrap starting.
I20260809 13:14:46.105911 12322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:46.107029 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:46.096142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:33781 (local address 127.0.0.1:33885)
0809 13:14:46.096949 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:46.096961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:46.097001 (+ 40us) server_negotiation.cc:407] Connection header received
0809 13:14:46.098646 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:46.098668 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:46.099056 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:46.099358 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:46.099986 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:46.101334 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:46.102775 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:46.103441 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:46.103604 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:46.104832 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:46.104840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:46.104853 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:46.105409 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:46.105518 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:46.105545 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:46.105690 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:46.105967 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 13:14:46.106114 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":394,"threads_started":1}
I20260809 13:14:46.107023 12322 log.cc:824] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:46.109875 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302: No bootstrap required, opened a new log
I20260809 13:14:46.110532 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:46.117909 12322 raft_consensus.cc:348] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } }
I20260809 13:14:46.118362 12322 raft_consensus.cc:374] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:46.118480 12322 raft_consensus.cc:729] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30cda7d99784457ace4da5685a87302, State: Initialized, Role: FOLLOWER
I20260809 13:14:46.119081 12322 consensus_queue.cc:260] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } }
I20260809 13:14:46.119316 12322 raft_consensus.cc:388] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:46.119509 12322 raft_consensus.cc:482] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:46.119719 12322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:46.121399 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:46.114796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:36901 (local address 127.0.0.1:33885)
0809 13:14:46.114982 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:46.114991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:46.115326 (+ 335us) server_negotiation.cc:407] Connection header received
0809 13:14:46.115519 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:46.115523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:46.115584 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:46.115665 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:46.116267 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:46.117029 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:46.118836 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:46.119328 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:46.119463 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:46.120632 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:46.120637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:46.120640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:46.120663 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:46.120722 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:46.120741 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:46.120830 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:46.121018 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 13:14:46.121069 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:14:46.122557 12322 raft_consensus.cc:504] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } }
I20260809 13:14:46.123169 12322 leader_election.cc:302] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c30cda7d99784457ace4da5685a87302; no voters:
I20260809 13:14:46.123984 12322 leader_election.cc:288] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:46.124115 12340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:46.124745 12340 raft_consensus.cc:686] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [term 1 LEADER]: Becoming Leader. State: Replica: c30cda7d99784457ace4da5685a87302, State: Running, Role: LEADER
I20260809 13:14:46.125268 12340 consensus_queue.cc:237] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } }
I20260809 13:14:46.125994 12322 sys_catalog.cc:564] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:46.128188 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c30cda7d99784457ace4da5685a87302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } } }
I20260809 13:14:46.128600 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:46.129339 12346 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c30cda7d99784457ace4da5685a87302. Latest consensus state: current_term: 1 leader_uuid: "c30cda7d99784457ace4da5685a87302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30cda7d99784457ace4da5685a87302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33885 } } }
I20260809 13:14:46.129732 12346 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:46.130223 12352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:46.135900 12352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:46.144699 12352 catalog_manager.cc:1347] Generated new cluster ID: 620dc0135def4f71a948102e65266bfe
I20260809 13:14:46.144798 12352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:46.157225 12352 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:46.158381 12352 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:46.173568 12352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c30cda7d99784457ace4da5685a87302: Generated new TSK 0
I20260809 13:14:46.174264 12352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:46.181787 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12212
W20260809 13:14:46.182928 12312 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.33:33781
W20260809 13:14:46.183600 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12212: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:46.183749 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12212
W20260809 13:14:46.213317 12190 connection.cc:535] client connection to 127.0.0.1:33885 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33885): Connection reset by peer (error 104)
2026-08-09T13:14:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:46 dist-test-slave-42z9 krb5kdc[12375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:46 dist-test-slave-42z9 krb5kdc[12375](info): set up 2 sockets
Aug 09 13:14:46 dist-test-slave-42z9 krb5kdc[12375](info): commencing operation
krb5kdc: starting...
W20260809 13:14:47.417147 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:47 dist-test-slave-42z9 krb5kdc[12375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 13:14:47.744879 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:48.080368 12393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:48.080664 12393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:48.080809 12393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:48.087860 12393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:48.087972 12393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:48.088050 12393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:48.088140 12393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:48.088198 12393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:48.097635 12393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:48.098866 12393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:48.100167 12393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:48.111296 12406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:48.111400 12403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:48.111343 12402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:48.112287 12393 server_base.cc:1034] running on GCE node
I20260809 13:14:48.112962 12393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:48.114451 12393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:48.115675 12393 hybrid_clock.cc:648] HybridClock initialized: now 1786281288115633 us; error 29 us; skew 500 ppm
I20260809 13:14:48.116040 12393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:48 dist-test-slave-42z9 krb5kdc[12375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281288, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:48.119143 12393 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 13:14:48.120545 12393 webserver.cc:460] Webserver started at http://127.0.0.1:36419/ using document root <none> and password file <none>
I20260809 13:14:48.121222 12393 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:48.121349 12393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:48.121692 12393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:48.124640 12393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "7c89696e67f84b3289778bd1121f134c"
format_stamp: "Formatted at 2026-08-09 13:14:48 on dist-test-slave-42z9"
server_key: "bdcc4b7d40a3fa9397f83473314076f9"
server_key_iv: "7c9c25af4feb966ce55bf1dfdabd59df"
server_key_version: "encryptionkey@0"
I20260809 13:14:48.125377 12393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "7c89696e67f84b3289778bd1121f134c"
format_stamp: "Formatted at 2026-08-09 13:14:48 on dist-test-slave-42z9"
server_key: "bdcc4b7d40a3fa9397f83473314076f9"
server_key_iv: "7c9c25af4feb966ce55bf1dfdabd59df"
server_key_version: "encryptionkey@0"
I20260809 13:14:48.130332 12393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 13:14:48.133613 12415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:48.134840 12393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:14:48.134991 12393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "7c89696e67f84b3289778bd1121f134c"
format_stamp: "Formatted at 2026-08-09 13:14:48 on dist-test-slave-42z9"
server_key: "bdcc4b7d40a3fa9397f83473314076f9"
server_key_iv: "7c9c25af4feb966ce55bf1dfdabd59df"
server_key_version: "encryptionkey@0"
I20260809 13:14:48.135172 12393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:48.140666 12393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:48.144353 12393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:48.144629 12393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:48.162154 12393 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44497
I20260809 13:14:48.162213 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44497 every 8 connection(s)
I20260809 13:14:48.163537 12393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:48.167162 12501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:48.173774 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12393
I20260809 13:14:48.173941 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:48.174361 10966 external_mini_cluster.cc:1428] Setting key 97e661576a89d0b9bdd21e591b6a5cd3
I20260809 13:14:48.177414 12501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c: Bootstrap starting.
I20260809 13:14:48.180995 12501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:48.181913 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:48.175993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:52103 (local address 127.0.0.1:44497)
0809 13:14:48.176746 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:48.176756 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:48.176798 (+ 42us) server_negotiation.cc:407] Connection header received
0809 13:14:48.178621 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:48.178645 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:48.178971 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:48.179275 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:48.180336 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:48.180342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:48.180355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:48.180369 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:48.180429 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:48.180467 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:48.180605 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:48.180757 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 13:14:48.180936 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":305,"threads_started":1}
I20260809 13:14:48.182166 12501 log.cc:824] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:48.185164 12501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c: No bootstrap required, opened a new log
I20260809 13:14:48.185567 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:48.192061 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:48.189756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:50039 (local address 127.0.0.1:44497)
0809 13:14:48.189937 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:48.189942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:48.190293 (+ 351us) server_negotiation.cc:407] Connection header received
0809 13:14:48.190456 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:48.190460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:48.190504 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:48.190586 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:48.191465 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:48.191469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:48.191472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:48.191489 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:48.191526 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:48.191540 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:48.191633 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:48.191753 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 13:14:48.191801 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:14:48.193339 12501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } }
I20260809 13:14:48.193738 12501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:48.193830 12501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c89696e67f84b3289778bd1121f134c, State: Initialized, Role: FOLLOWER
I20260809 13:14:48.194401 12501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } }
I20260809 13:14:48.194641 12501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:48.194783 12501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:48.194965 12501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:48.197088 12501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } }
I20260809 13:14:48.197587 12501 leader_election.cc:302] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c89696e67f84b3289778bd1121f134c; no voters:
I20260809 13:14:48.198104 12501 leader_election.cc:288] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:48.198242 12514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:48.198860 12514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [term 1 LEADER]: Becoming Leader. State: Replica: 7c89696e67f84b3289778bd1121f134c, State: Running, Role: LEADER
I20260809 13:14:48.199460 12514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } }
I20260809 13:14:48.200145 12501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:48.202448 12517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c89696e67f84b3289778bd1121f134c. Latest consensus state: current_term: 1 leader_uuid: "7c89696e67f84b3289778bd1121f134c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } } }
I20260809 13:14:48.202904 12517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:48.202790 12516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c89696e67f84b3289778bd1121f134c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89696e67f84b3289778bd1121f134c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44497 } } }
I20260809 13:14:48.203081 12516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:48.204088 12525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:48.209105 12525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:48.217918 12525 catalog_manager.cc:1347] Generated new cluster ID: 40faed19a47c401696b17f7871b9881e
I20260809 13:14:48.218058 12525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:48.233512 12525 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:48.233974 12525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:48.241253 12525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c89696e67f84b3289778bd1121f134c: Generated new TSK 0
I20260809 13:14:48.242225 12525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:48.250116 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12393
W20260809 13:14:48.251547 12494 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.33:52103
W20260809 13:14:48.252290 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12393: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:48.252447 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12393
2026-08-09T13:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:48 dist-test-slave-42z9 krb5kdc[12543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:48 dist-test-slave-42z9 krb5kdc[12543](info): set up 2 sockets
Aug 09 13:14:48 dist-test-slave-42z9 krb5kdc[12543](info): commencing operation
krb5kdc: starting...
W20260809 13:14:49.555371 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:49 dist-test-slave-42z9 krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:49.844559 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36745
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:36745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:50.174289 12564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:50.174559 12564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:50.174692 12564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:50.181680 12564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:50.181777 12564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:50.181851 12564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:50.181929 12564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:50.181991 12564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:50.191306 12564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:36745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:36745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:50.192482 12564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:50.193832 12564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:50.204823 12569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:50.204871 12570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:50.204910 12578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:50.205628 12564 server_base.cc:1034] running on GCE node
I20260809 13:14:50.206352 12564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:50.207634 12564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:50.208832 12564 hybrid_clock.cc:648] HybridClock initialized: now 1786281290208792 us; error 28 us; skew 500 ppm
I20260809 13:14:50.209156 12564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:50 dist-test-slave-42z9 krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281290, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:50.212205 12564 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:50.213610 12564 webserver.cc:460] Webserver started at http://127.10.181.190:43039/ using document root <none> and password file <none>
I20260809 13:14:50.214241 12564 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:50.214359 12564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:50.214649 12564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:50.217494 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "be6d682bf69345ffb422179e38c04948"
format_stamp: "Formatted at 2026-08-09 13:14:50 on dist-test-slave-42z9"
server_key: "3c03e2b96d00a8d81ae70441f9bc0ced"
server_key_iv: "e779de0400ba9ac1cde9239eee119840"
server_key_version: "encryptionkey@0"
I20260809 13:14:50.218232 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "be6d682bf69345ffb422179e38c04948"
format_stamp: "Formatted at 2026-08-09 13:14:50 on dist-test-slave-42z9"
server_key: "3c03e2b96d00a8d81ae70441f9bc0ced"
server_key_iv: "e779de0400ba9ac1cde9239eee119840"
server_key_version: "encryptionkey@0"
I20260809 13:14:50.223021 12564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 13:14:50.226310 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:50.227499 12564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 13:14:50.227658 12564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "be6d682bf69345ffb422179e38c04948"
format_stamp: "Formatted at 2026-08-09 13:14:50 on dist-test-slave-42z9"
server_key: "3c03e2b96d00a8d81ae70441f9bc0ced"
server_key_iv: "e779de0400ba9ac1cde9239eee119840"
server_key_version: "encryptionkey@0"
I20260809 13:14:50.227828 12564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:50.252270 12564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:50.256008 12564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:50.256264 12564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:50.273603 12564 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:36745
I20260809 13:14:50.273628 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:36745 every 8 connection(s)
I20260809 13:14:50.274943 12564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:50.278604 12672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:50.280330 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12564
I20260809 13:14:50.280520 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:50.281061 10966 external_mini_cluster.cc:1428] Setting key 1629c893472a82f230cd2e6bd39626c7
I20260809 13:14:50.291544 12672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948: Bootstrap starting.
Aug 09 13:14:50 dist-test-slave-42z9 krb5kdc[12543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:50.294847 12672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:50.295878 12672 log.cc:824] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:50.298889 12672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948: No bootstrap required, opened a new log
I20260809 13:14:50.302894 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:50.283012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47917 (local address 127.10.181.190:36745)
0809 13:14:50.283815 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:50.283826 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:50.283867 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:50.285433 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:50.285487 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:50.285881 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:50.286350 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:50.287755 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:50.289092 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:50.290424 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:50.291074 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:50.291240 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:50.295621 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:50.295631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:50.295646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:50.295682 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:50.298821 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:50.299638 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:50.299645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:50.299650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:50.299814 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:50.300256 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:50.300260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:50.300265 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:50.300806 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:50.301145 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:50.301623 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 13:14:50.301931 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":365,"threads_started":1}
I20260809 13:14:50.306773 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:50.307333 12672 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } }
I20260809 13:14:50.307678 12672 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:50.307768 12672 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6d682bf69345ffb422179e38c04948, State: Initialized, Role: FOLLOWER
I20260809 13:14:50.308279 12672 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } }
I20260809 13:14:50.308498 12672 raft_consensus.cc:388] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:50.308630 12672 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:50.308774 12672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:50.311115 12672 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } }
I20260809 13:14:50.311679 12672 leader_election.cc:302] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be6d682bf69345ffb422179e38c04948; no voters:
I20260809 13:14:50.312276 12672 leader_election.cc:288] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:50.312417 12691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:50.313702 12691 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [term 1 LEADER]: Becoming Leader. State: Replica: be6d682bf69345ffb422179e38c04948, State: Running, Role: LEADER
I20260809 13:14:50.314393 12691 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } }
I20260809 13:14:50.314507 12672 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:50.317076 12692 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be6d682bf69345ffb422179e38c04948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } } }
I20260809 13:14:50.317544 12692 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:50.317111 12695 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6d682bf69345ffb422179e38c04948. Latest consensus state: current_term: 1 leader_uuid: "be6d682bf69345ffb422179e38c04948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6d682bf69345ffb422179e38c04948" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 36745 } } }
I20260809 13:14:50.321403 12695 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:50.323727 12700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:50.323923 12714 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 13:14:50.328907 12700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:50.333639 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:50.311129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39325 (local address 127.10.181.190:36745)
0809 13:14:50.311312 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:50.311318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:50.311905 (+ 587us) server_negotiation.cc:407] Connection header received
0809 13:14:50.312060 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:50.312070 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:50.312162 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:50.312287 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:50.314219 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:50.315355 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:50.322299 (+ 6944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:50.322995 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:50.323137 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:50.325887 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:50.325894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:50.325898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:50.325946 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:50.329754 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:50.330761 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:50.330766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:50.330770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:50.330924 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:50.331609 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:50.331743 (+ 134us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:50.331747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:50.332157 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:50.332463 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:50.333096 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 13:14:50.333263 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 13:14:50.338126 12700 catalog_manager.cc:1347] Generated new cluster ID: 4aa70cb8d88a4c5c823d73308d67952d
I20260809 13:14:50.338227 12700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:50.355859 12700 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:50.357141 12700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:50.369510 12700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be6d682bf69345ffb422179e38c04948: Generated new TSK 0
I20260809 13:14:50.370338 12700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:50.379401 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12564
I20260809 13:14:50.380425 12665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:50.473832 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12564
W20260809 13:14:50.499415 12541 connection.cc:535] client connection to 127.10.181.190:36745 recv error: Network error: failed to read from TLS socket (remote: 127.10.181.190:36745): Connection reset by peer (error 104)
2026-08-09T13:14:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:50 dist-test-slave-42z9 krb5kdc[12723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:50 dist-test-slave-42z9 krb5kdc[12723](info): set up 2 sockets
Aug 09 13:14:50 dist-test-slave-42z9 krb5kdc[12723](info): commencing operation
krb5kdc: starting...
W20260809 13:14:51.702550 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:51 dist-test-slave-42z9 krb5kdc[12723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:52.004159 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33341
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:33341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:52.335131 12741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:52.335423 12741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:52.335556 12741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:52.342450 12741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:52.342556 12741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:52.342633 12741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:52.342710 12741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:52.342767 12741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:52.352020 12741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:33341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:33341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:52.353191 12741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:52.354485 12741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:52.365289 12751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:52.365334 12748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:52.365350 12747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:52.366125 12741 server_base.cc:1034] running on GCE node
I20260809 13:14:52.366814 12741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:52.368083 12741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:52.369275 12741 hybrid_clock.cc:648] HybridClock initialized: now 1786281292369231 us; error 39 us; skew 500 ppm
I20260809 13:14:52.369623 12741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:52 dist-test-slave-42z9 krb5kdc[12723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:52.372519 12741 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:52.373898 12741 webserver.cc:460] Webserver started at http://127.10.181.190:36761/ using document root <none> and password file <none>
I20260809 13:14:52.374516 12741 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:52.374614 12741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:52.374852 12741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:52.377702 12741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "ba8e57c71df041b7ab7722798859b853"
format_stamp: "Formatted at 2026-08-09 13:14:52 on dist-test-slave-42z9"
server_key: "fa8b575dd1bc52d2dfa299da4cff6211"
server_key_iv: "2ad36407d9ae58177ecfac09ea341f09"
server_key_version: "encryptionkey@0"
I20260809 13:14:52.378422 12741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "ba8e57c71df041b7ab7722798859b853"
format_stamp: "Formatted at 2026-08-09 13:14:52 on dist-test-slave-42z9"
server_key: "fa8b575dd1bc52d2dfa299da4cff6211"
server_key_iv: "2ad36407d9ae58177ecfac09ea341f09"
server_key_version: "encryptionkey@0"
I20260809 13:14:52.383234 12741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 13:14:52.386492 12763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:52.387732 12741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:14:52.387887 12741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "ba8e57c71df041b7ab7722798859b853"
format_stamp: "Formatted at 2026-08-09 13:14:52 on dist-test-slave-42z9"
server_key: "fa8b575dd1bc52d2dfa299da4cff6211"
server_key_iv: "2ad36407d9ae58177ecfac09ea341f09"
server_key_version: "encryptionkey@0"
I20260809 13:14:52.388055 12741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:52.404533 12741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:52.407996 12741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:52.408232 12741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:52.425019 12741 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:33341
I20260809 13:14:52.425061 12844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:33341 every 8 connection(s)
I20260809 13:14:52.426352 12741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:52.428946 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12741
I20260809 13:14:52.429136 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:52.429704 10966 external_mini_cluster.cc:1428] Setting key d0a17d77fb9678f8f588b3f066d5483b
I20260809 13:14:52.430299 12845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:52.442334 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:52.431414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37593 (local address 127.10.181.190:33341)
0809 13:14:52.432146 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:52.432157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:52.432196 (+ 39us) server_negotiation.cc:407] Connection header received
0809 13:14:52.433714 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:52.433735 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:52.434100 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:52.434412 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:52.435011 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:52.436313 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:52.438125 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:52.438764 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:52.438888 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:52.440174 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:52.440186 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:52.440198 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:52.440721 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:52.440796 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:52.440821 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:52.440948 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:52.441234 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 13:14:52.441415 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":327,"threads_started":1}
I20260809 13:14:52.443439 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853: Bootstrap starting.
I20260809 13:14:52.445827 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:52.446591 12845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:52.447738 12845 log.cc:824] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:52.450405 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853: No bootstrap required, opened a new log
I20260809 13:14:52.456972 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:52.450717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60381 (local address 127.10.181.190:33341)
0809 13:14:52.450903 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:52.450908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:52.451263 (+ 355us) server_negotiation.cc:407] Connection header received
0809 13:14:52.451403 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:52.451408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:52.451478 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:52.451586 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:52.452026 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:52.453172 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:52.454433 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:52.455102 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:52.455246 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:52.456254 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:52.456260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:52.456265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:52.456285 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:52.456350 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:52.456371 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:52.456488 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:52.456588 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 13:14:52.456642 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:14:52.459287 12845 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } }
I20260809 13:14:52.459632 12845 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:52.459707 12845 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8e57c71df041b7ab7722798859b853, State: Initialized, Role: FOLLOWER
I20260809 13:14:52.460209 12845 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } }
I20260809 13:14:52.460417 12845 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:52.460515 12845 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:52.460656 12845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:52.462832 12845 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } }
I20260809 13:14:52.463302 12845 leader_election.cc:302] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba8e57c71df041b7ab7722798859b853; no voters:
I20260809 13:14:52.463851 12845 leader_election.cc:288] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:52.463959 12859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:52.464627 12859 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [term 1 LEADER]: Becoming Leader. State: Replica: ba8e57c71df041b7ab7722798859b853, State: Running, Role: LEADER
I20260809 13:14:52.465157 12859 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } }
I20260809 13:14:52.465757 12845 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:52.468575 12860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba8e57c71df041b7ab7722798859b853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } } }
I20260809 13:14:52.468971 12860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:52.469835 12875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:52.469700 12865 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba8e57c71df041b7ab7722798859b853. Latest consensus state: current_term: 1 leader_uuid: "ba8e57c71df041b7ab7722798859b853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba8e57c71df041b7ab7722798859b853" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 33341 } } }
I20260809 13:14:52.470095 12865 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:52.475109 12875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:52.484048 12875 catalog_manager.cc:1347] Generated new cluster ID: b14da43c8be843d18636ca8395966fc7
I20260809 13:14:52.484181 12875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:52.490549 12875 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:52.491744 12875 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:52.504163 12875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba8e57c71df041b7ab7722798859b853: Generated new TSK 0
I20260809 13:14:52.505085 12875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:52.515688 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12741
W20260809 13:14:52.516776 12835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37593
W20260809 13:14:52.517505 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12741: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:52.517673 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12741
W20260809 13:14:52.550020 12721 connection.cc:535] client connection to 127.10.181.190:33341 recv error: Network error: failed to read from TLS socket (remote: 127.10.181.190:33341): Connection reset by peer (error 104)
2026-08-09T13:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:52 dist-test-slave-42z9 krb5kdc[12895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:52 dist-test-slave-42z9 krb5kdc[12895](info): set up 2 sockets
Aug 09 13:14:52 dist-test-slave-42z9 krb5kdc[12895](info): commencing operation
krb5kdc: starting...
W20260809 13:14:53.759859 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:53 dist-test-slave-42z9 krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:54.064894 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40413
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:40413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:54.400817 12915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:54.401094 12915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:54.401225 12915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:54.408125 12915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:54.408227 12915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:54.408310 12915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:54.408385 12915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:54.408443 12915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:54.418192 12915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:40413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:54.419420 12915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:54.420675 12915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:54.431888 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:54.431929 12925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:54.432031 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:54.432628 12915 server_base.cc:1034] running on GCE node
I20260809 13:14:54.433420 12915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:54.434747 12915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:54.435979 12915 hybrid_clock.cc:648] HybridClock initialized: now 1786281294435937 us; error 36 us; skew 500 ppm
I20260809 13:14:54.436362 12915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:54 dist-test-slave-42z9 krb5kdc[12895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281294, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:54.439471 12915 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:54.440912 12915 webserver.cc:460] Webserver started at http://127.10.181.190:34277/ using document root <none> and password file <none>
I20260809 13:14:54.441624 12915 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:54.441746 12915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:54.442034 12915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:54.444968 12915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "5ae33725cc374201897307ad5840b5a9"
format_stamp: "Formatted at 2026-08-09 13:14:54 on dist-test-slave-42z9"
server_key: "387e64676fb34c4089602c177e4206d7"
server_key_iv: "600e836c5bbe5039c110de835a60af2b"
server_key_version: "encryptionkey@0"
I20260809 13:14:54.445775 12915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "5ae33725cc374201897307ad5840b5a9"
format_stamp: "Formatted at 2026-08-09 13:14:54 on dist-test-slave-42z9"
server_key: "387e64676fb34c4089602c177e4206d7"
server_key_iv: "600e836c5bbe5039c110de835a60af2b"
server_key_version: "encryptionkey@0"
I20260809 13:14:54.450704 12915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 13:14:54.454052 12934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:54.455272 12915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:14:54.455431 12915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "5ae33725cc374201897307ad5840b5a9"
format_stamp: "Formatted at 2026-08-09 13:14:54 on dist-test-slave-42z9"
server_key: "387e64676fb34c4089602c177e4206d7"
server_key_iv: "600e836c5bbe5039c110de835a60af2b"
server_key_version: "encryptionkey@0"
I20260809 13:14:54.455605 12915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:54.461315 12915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:54.465412 12915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:54.465718 12915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:54.483057 12915 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:40413
I20260809 13:14:54.483091 13013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:40413 every 8 connection(s)
I20260809 13:14:54.484375 12915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:54.488073 13015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:54.494029 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 12915
I20260809 13:14:54.494190 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:54.494611 10966 external_mini_cluster.cc:1428] Setting key 12544e4d4599666aa34a063d54682cfd
I20260809 13:14:54.498585 13015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9: Bootstrap starting.
I20260809 13:14:54.502054 13015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:54.502137 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:54.496346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50823 (local address 127.10.181.190:40413)
0809 13:14:54.497189 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:54.497200 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:54.497241 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:54.498884 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:54.498907 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:54.499216 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:54.499515 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:54.500626 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:54.500633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:54.500645 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:54.500660 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:54.500716 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:54.500737 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:54.500895 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:54.501033 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 13:14:54.501185 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":364,"threads_started":1}
I20260809 13:14:54.503127 13015 log.cc:824] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:54.505935 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:54.506378 13015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9: No bootstrap required, opened a new log
I20260809 13:14:54.513248 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:54.510880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38611 (local address 127.10.181.190:40413)
0809 13:14:54.511070 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:54.511076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:54.511507 (+ 431us) server_negotiation.cc:407] Connection header received
0809 13:14:54.511666 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:54.511670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:54.511723 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:54.511798 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:54.512665 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:54.512669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:54.512672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:54.512686 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:54.512722 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:54.512739 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:54.512837 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:54.512933 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 13:14:54.512980 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:14:54.515774 13015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } }
I20260809 13:14:54.516100 13015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:54.516175 13015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae33725cc374201897307ad5840b5a9, State: Initialized, Role: FOLLOWER
I20260809 13:14:54.516690 13015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } }
I20260809 13:14:54.516903 13015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:54.517004 13015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:54.517140 13015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:54.519486 13015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } }
I20260809 13:14:54.519991 13015 leader_election.cc:302] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ae33725cc374201897307ad5840b5a9; no voters:
I20260809 13:14:54.520537 13015 leader_election.cc:288] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:54.520648 13031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:54.521263 13031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae33725cc374201897307ad5840b5a9, State: Running, Role: LEADER
I20260809 13:14:54.521771 13031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } }
I20260809 13:14:54.522483 13015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:54.524448 13033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ae33725cc374201897307ad5840b5a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } } }
I20260809 13:14:54.524880 13033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:54.525385 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ae33725cc374201897307ad5840b5a9. Latest consensus state: current_term: 1 leader_uuid: "5ae33725cc374201897307ad5840b5a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae33725cc374201897307ad5840b5a9" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40413 } } }
I20260809 13:14:54.525707 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:54.527313 13046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:54.532382 13046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:54.541178 13046 catalog_manager.cc:1347] Generated new cluster ID: 0bd00856adab4d2d8aa9dd71eb987b4a
I20260809 13:14:54.541267 13046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:54.553797 13046 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:54.554157 13046 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:54.563475 13046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ae33725cc374201897307ad5840b5a9: Generated new TSK 0
I20260809 13:14:54.564185 13046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:54.580104 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12915
W20260809 13:14:54.581269 13007 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50823
W20260809 13:14:54.581956 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 12915: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:54.582079 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 12915
2026-08-09T13:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:54 dist-test-slave-42z9 krb5kdc[13060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:54 dist-test-slave-42z9 krb5kdc[13060](info): set up 2 sockets
Aug 09 13:14:54 dist-test-slave-42z9 krb5kdc[13060](info): commencing operation
krb5kdc: starting...
W20260809 13:14:55.821252 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:56.118839 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34359
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:43513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:34359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:56.452210 13084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:56.452507 13084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:56.452648 13084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:56.459640 13084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:56.459738 13084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:56.459815 13084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:56.459906 13084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:56.459964 13084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:56.469365 13084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:43513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:34359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:34359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:56.470594 13084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:56.471850 13084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:56.483134 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:56.483198 13095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:56.483170 13098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:56.484153 13084 server_base.cc:1034] running on GCE node
I20260809 13:14:56.484934 13084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:56.486431 13084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:56.487641 13084 hybrid_clock.cc:648] HybridClock initialized: now 1786281296487583 us; error 47 us; skew 500 ppm
I20260809 13:14:56.487983 13084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:56.490932 13084 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:56.492357 13084 webserver.cc:460] Webserver started at http://127.10.181.190:43359/ using document root <none> and password file <none>
I20260809 13:14:56.493014 13084 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:56.493134 13084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:56.493477 13084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:56.496392 13084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "30a73686f2c54ee49d4104d00541bd09"
format_stamp: "Formatted at 2026-08-09 13:14:56 on dist-test-slave-42z9"
server_key: "fef4683da63e8f859b4a1bbc7f405b5d"
server_key_iv: "0e0628dabd5e7f2f7415a74914ef4fc9"
server_key_version: "encryptionkey@0"
I20260809 13:14:56.497126 13084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "30a73686f2c54ee49d4104d00541bd09"
format_stamp: "Formatted at 2026-08-09 13:14:56 on dist-test-slave-42z9"
server_key: "fef4683da63e8f859b4a1bbc7f405b5d"
server_key_iv: "0e0628dabd5e7f2f7415a74914ef4fc9"
server_key_version: "encryptionkey@0"
I20260809 13:14:56.502167 13084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 13:14:56.505496 13107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:56.506767 13084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 13:14:56.506927 13084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "30a73686f2c54ee49d4104d00541bd09"
format_stamp: "Formatted at 2026-08-09 13:14:56 on dist-test-slave-42z9"
server_key: "fef4683da63e8f859b4a1bbc7f405b5d"
server_key_iv: "0e0628dabd5e7f2f7415a74914ef4fc9"
server_key_version: "encryptionkey@0"
I20260809 13:14:56.507104 13084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:56.523311 13084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:56.527057 13084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:56.527320 13084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:56.544564 13084 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:34359
I20260809 13:14:56.544571 13179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:34359 every 8 connection(s)
I20260809 13:14:56.545862 13084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:56.546319 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13084
I20260809 13:14:56.546489 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:56.547079 10966 external_mini_cluster.cc:1428] Setting key d4de42178c14a5afb1603196556a7177
I20260809 13:14:56.550004 13182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:14:56.560446 13182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09: Bootstrap starting.
I20260809 13:14:56.563784 13182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:56.564949 13182 log.cc:824] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:56.567992 13182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09: No bootstrap required, opened a new log
I20260809 13:14:56.568331 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:56.548836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60581 (local address 127.10.181.190:34359)
0809 13:14:56.549631 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:56.549643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:56.549684 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:14:56.551311 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:56.551333 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:56.551697 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:56.552029 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:56.553559 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:56.554899 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:56.556128 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:56.556761 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:56.556938 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:56.561231 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:56.561238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:56.561251 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:56.561297 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:56.564214 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:56.564987 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:56.564993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:56.564999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:56.565195 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:56.565665 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:56.565669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:56.565673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:56.566371 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:56.566593 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:56.567032 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 13:14:56.567359 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":333,"threads_started":1}
I20260809 13:14:56.572041 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:56.577699 13182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } }
I20260809 13:14:56.578025 13182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:56.578114 13182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a73686f2c54ee49d4104d00541bd09, State: Initialized, Role: FOLLOWER
I20260809 13:14:56.578656 13182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } }
I20260809 13:14:56.578874 13182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:56.579005 13182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:56.579155 13182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:56.581192 13182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } }
I20260809 13:14:56.581694 13182 leader_election.cc:302] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30a73686f2c54ee49d4104d00541bd09; no voters:
I20260809 13:14:56.582267 13182 leader_election.cc:288] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:56.582376 13196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:56.584393 13182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:56.588316 13196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [term 1 LEADER]: Becoming Leader. State: Replica: 30a73686f2c54ee49d4104d00541bd09, State: Running, Role: LEADER
I20260809 13:14:56.589700 13196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } }
I20260809 13:14:56.592716 13200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30a73686f2c54ee49d4104d00541bd09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } } }
I20260809 13:14:56.593113 13200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:56.593657 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30a73686f2c54ee49d4104d00541bd09. Latest consensus state: current_term: 1 leader_uuid: "30a73686f2c54ee49d4104d00541bd09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a73686f2c54ee49d4104d00541bd09" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 34359 } } }
I20260809 13:14:56.593921 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09 [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:56.599555 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:56.577547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51699 (local address 127.10.181.190:34359)
0809 13:14:56.577745 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:56.577751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:56.584007 (+ 6256us) server_negotiation.cc:407] Connection header received
0809 13:14:56.584198 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:56.584203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:56.584281 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:56.584366 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:56.585925 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:56.586658 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:56.588165 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:56.588874 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:56.588992 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:56.591878 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:56.591884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:56.591889 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:14:56.591930 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:56.595442 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:56.596333 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:56.596340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:56.596344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:56.596585 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:14:56.597077 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:14:56.597082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:14:56.597086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:14:56.597498 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:56.597797 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:56.598925 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0809 13:14:56.599149 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:14:56.604089 13217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:56.608212 13217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:56.616449 13217 catalog_manager.cc:1347] Generated new cluster ID: ea8e4f5e6be84ebc9aff2c8b5f7a46e1
I20260809 13:14:56.616546 13217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:56.637750 13217 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:56.638988 13217 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:56.644711 13217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30a73686f2c54ee49d4104d00541bd09: Generated new TSK 0
I20260809 13:14:56.645412 13217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:56.651531 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13084
I20260809 13:14:56.652470 13173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:14:56.742041 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13084
2026-08-09T13:14:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2159 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13229](info): set up 2 sockets
Aug 09 13:14:56 dist-test-slave-42z9 krb5kdc[13229](info): commencing operation
krb5kdc: starting...
W20260809 13:14:57.977633 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:14:58 dist-test-slave-42z9 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:14:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:14:58.296676 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40555
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:40555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:14:58.631973 13248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:14:58.632256 13248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:14:58.632405 13248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:14:58.639366 13248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:14:58.639501 13248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:14:58.639577 13248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:14:58.639658 13248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:14:58.639726 13248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:14:58.649086 13248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:40555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:14:58.650332 13248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:14:58.651638 13248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:14:58.663058 13257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:14:58.663098 13262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:58.663705 13248 server_base.cc:1034] running on GCE node
W20260809 13:14:58.663674 13260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:14:58.664529 13248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:14:58.665860 13248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:14:58.667057 13248 hybrid_clock.cc:648] HybridClock initialized: now 1786281298667015 us; error 31 us; skew 500 ppm
I20260809 13:14:58.667374 13248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:14:58 dist-test-slave-42z9 krb5kdc[13229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:14:58.673601 13248 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:14:58.674932 13248 webserver.cc:460] Webserver started at http://127.10.181.190:38607/ using document root <none> and password file <none>
I20260809 13:14:58.675550 13248 fs_manager.cc:359] Metadata directory not provided
I20260809 13:14:58.675649 13248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:14:58.675891 13248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:14:58.678771 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "3a36c1c8daa045dca79957a9ed2c8dfa"
format_stamp: "Formatted at 2026-08-09 13:14:58 on dist-test-slave-42z9"
server_key: "841e72cdf48c37ecbce056dbf420758f"
server_key_iv: "404887c1b36a662dfd839369b83d03c7"
server_key_version: "encryptionkey@0"
I20260809 13:14:58.679497 13248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "3a36c1c8daa045dca79957a9ed2c8dfa"
format_stamp: "Formatted at 2026-08-09 13:14:58 on dist-test-slave-42z9"
server_key: "841e72cdf48c37ecbce056dbf420758f"
server_key_iv: "404887c1b36a662dfd839369b83d03c7"
server_key_version: "encryptionkey@0"
I20260809 13:14:58.684331 13248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 13:14:58.687657 13273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:14:58.688930 13248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:14:58.689095 13248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "3a36c1c8daa045dca79957a9ed2c8dfa"
format_stamp: "Formatted at 2026-08-09 13:14:58 on dist-test-slave-42z9"
server_key: "841e72cdf48c37ecbce056dbf420758f"
server_key_iv: "404887c1b36a662dfd839369b83d03c7"
server_key_version: "encryptionkey@0"
I20260809 13:14:58.689277 13248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:14:58.722993 13248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:14:58.726969 13248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:14:58.727226 13248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:14:58.744310 13248 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:40555
I20260809 13:14:58.744330 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:40555 every 8 connection(s)
I20260809 13:14:58.745659 13248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:14:58.749193 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:14:58.754187 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13248
I20260809 13:14:58.754366 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:14:58.754957 10966 external_mini_cluster.cc:1428] Setting key ae3458e7dea61dc696ca7cf1de0a5fa5
I20260809 13:14:58.760046 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa: Bootstrap starting.
I20260809 13:14:58.763481 13359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa: Neither blocks nor log segments found. Creating new log.
I20260809 13:14:58.764567 13359 log.cc:824] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa: Log is configured to *not* fsync() on all Append() calls
I20260809 13:14:58.767494 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa: No bootstrap required, opened a new log
I20260809 13:14:58.767856 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:58.756709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56853 (local address 127.10.181.190:40555)
0809 13:14:58.757573 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:58.757584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:58.757630 (+ 46us) server_negotiation.cc:407] Connection header received
0809 13:14:58.759336 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:58.759359 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:58.759725 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:58.760011 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:58.760626 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:58.761955 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:58.763734 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:58.764391 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:58.764517 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:58.765771 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:58.765782 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:58.765800 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:58.766322 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:58.766394 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:58.766418 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:58.766502 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:58.766721 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 13:14:58.766930 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":368,"threads_started":1}
I20260809 13:14:58.771281 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:14:58.776319 13359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } }
I20260809 13:14:58.776706 13359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:14:58.776803 13359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a36c1c8daa045dca79957a9ed2c8dfa, State: Initialized, Role: FOLLOWER
I20260809 13:14:58.777355 13359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } }
I20260809 13:14:58.777635 13359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:14:58.777791 13359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:14:58.777982 13359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:14:58.780453 13359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } }
I20260809 13:14:58.781035 13359 leader_election.cc:302] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a36c1c8daa045dca79957a9ed2c8dfa; no voters:
I20260809 13:14:58.781656 13359 leader_election.cc:288] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:14:58.781741 13376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:14:58.782522 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:14:58.776212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47321 (local address 127.10.181.190:40555)
0809 13:14:58.776382 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:14:58.776387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:14:58.776822 (+ 435us) server_negotiation.cc:407] Connection header received
0809 13:14:58.776987 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:14:58.776990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:14:58.777045 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:14:58.777137 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:14:58.777678 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:58.778703 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:58.779958 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:14:58.780679 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:14:58.780801 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:14:58.781774 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:14:58.781781 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:14:58.781786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:14:58.781806 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:14:58.781851 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:14:58.781875 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:14:58.781976 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 13:14:58.782097 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 13:14:58.782189 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 13:14:58.783478 13376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [term 1 LEADER]: Becoming Leader. State: Replica: 3a36c1c8daa045dca79957a9ed2c8dfa, State: Running, Role: LEADER
I20260809 13:14:58.784092 13376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } }
I20260809 13:14:58.784335 13359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:14:58.786986 13381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a36c1c8daa045dca79957a9ed2c8dfa. Latest consensus state: current_term: 1 leader_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } } }
I20260809 13:14:58.787348 13381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:58.787832 13380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a36c1c8daa045dca79957a9ed2c8dfa" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 40555 } } }
I20260809 13:14:58.788177 13380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa [sys.catalog]: This master's current role is: LEADER
I20260809 13:14:58.789949 13390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:14:58.794647 13390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:14:58.802918 13390 catalog_manager.cc:1347] Generated new cluster ID: c471b0c133e1450d874fe21502d3b2f5
I20260809 13:14:58.803023 13390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:14:58.813409 13390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:14:58.814635 13390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:14:58.833551 13390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a36c1c8daa045dca79957a9ed2c8dfa: Generated new TSK 0
I20260809 13:14:58.834316 13390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:14:58.844985 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13248
W20260809 13:14:58.845940 13348 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56853
W20260809 13:14:58.846573 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13248: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:14:58.846701 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13248
2026-08-09T13:14:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2103 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:14:58 dist-test-slave-42z9 krb5kdc[13411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:14:58 dist-test-slave-42z9 krb5kdc[13411](info): set up 2 sockets
Aug 09 13:14:58 dist-test-slave-42z9 krb5kdc[13411](info): commencing operation
krb5kdc: starting...
W20260809 13:15:00.091955 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:15:00 dist-test-slave-42z9 krb5kdc[13411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:15:00.422744 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46301
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:42385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:46301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:15:00.767428 13430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:00.767692 13430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:00.767791 13430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:00.774704 13430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:00.774806 13430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:00.774850 13430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:00.774917 13430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:00.774960 13430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:00.784373 13430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:42385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:46301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:46301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:00.785578 13430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:00.786857 13430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:15:00.798580 13435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:00.798628 13436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:00.798683 13441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:00.799360 13430 server_base.cc:1034] running on GCE node
I20260809 13:15:00.800145 13430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:00.801553 13430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:00.802772 13430 hybrid_clock.cc:648] HybridClock initialized: now 1786281300802720 us; error 46 us; skew 500 ppm
I20260809 13:15:00.803113 13430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:00 dist-test-slave-42z9 krb5kdc[13411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281300, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:00.806187 13430 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:15:00.807718 13430 webserver.cc:460] Webserver started at http://127.10.181.190:33961/ using document root <none> and password file <none>
I20260809 13:15:00.808483 13430 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:00.808609 13430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:00.808917 13430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:00.811980 13430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "250542ca5b7042dc839814329a5646f0"
format_stamp: "Formatted at 2026-08-09 13:15:00 on dist-test-slave-42z9"
server_key: "e9978f511e55d176b05d2c3b205a993e"
server_key_iv: "ff3269d11bb280f846ffb4548d79baa3"
server_key_version: "encryptionkey@0"
I20260809 13:15:00.812753 13430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "250542ca5b7042dc839814329a5646f0"
format_stamp: "Formatted at 2026-08-09 13:15:00 on dist-test-slave-42z9"
server_key: "e9978f511e55d176b05d2c3b205a993e"
server_key_iv: "ff3269d11bb280f846ffb4548d79baa3"
server_key_version: "encryptionkey@0"
I20260809 13:15:00.817880 13430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 13:15:00.821319 13452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:00.822651 13430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:15:00.822813 13430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "250542ca5b7042dc839814329a5646f0"
format_stamp: "Formatted at 2026-08-09 13:15:00 on dist-test-slave-42z9"
server_key: "e9978f511e55d176b05d2c3b205a993e"
server_key_iv: "ff3269d11bb280f846ffb4548d79baa3"
server_key_version: "encryptionkey@0"
I20260809 13:15:00.822992 13430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:00.828584 13430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:00.832712 13430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:00.832980 13430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:00.850484 13430 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:46301
I20260809 13:15:00.850549 13536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:46301 every 8 connection(s)
I20260809 13:15:00.851822 13430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:15:00.855449 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13430
I20260809 13:15:00.855513 13537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:00.855634 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:15:00.856176 10966 external_mini_cluster.cc:1428] Setting key c3bda57b347ffb5c9a7706110a70b314
I20260809 13:15:00.863466 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:00.857892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43055 (local address 127.10.181.190:46301)
0809 13:15:00.858749 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:00.858759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:00.858804 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:15:00.860386 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:00.860407 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:00.860721 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:00.861051 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:00.862056 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:00.862063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:00.862075 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:15:00.862089 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:00.862148 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:15:00.862169 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:00.862338 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:00.862435 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 13:15:00.862597 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":407,"threads_started":1}
I20260809 13:15:00.866815 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:15:00.868618 13537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0: Bootstrap starting.
I20260809 13:15:00.872341 13537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:00.873665 13537 log.cc:824] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:00.874172 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:00.871791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51789 (local address 127.10.181.190:46301)
0809 13:15:00.871915 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:00.871919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:00.872346 (+ 427us) server_negotiation.cc:407] Connection header received
0809 13:15:00.872547 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:00.872550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:00.872593 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:00.872674 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:00.873574 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:00.873578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:00.873581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 13:15:00.873595 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:00.873634 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 13:15:00.873648 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:00.873736 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:00.873885 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 13:15:00.873930 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 13:15:00.876307 13537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0: No bootstrap required, opened a new log
I20260809 13:15:00.883203 13537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } }
I20260809 13:15:00.883520 13537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 13:15:00.883592 13537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 250542ca5b7042dc839814329a5646f0, State: Initialized, Role: FOLLOWER
I20260809 13:15:00.884078 13537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } }
I20260809 13:15:00.884295 13537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 13:15:00.884433 13537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 13:15:00.884606 13537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:00.886690 13537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } }
I20260809 13:15:00.887142 13537 leader_election.cc:302] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 250542ca5b7042dc839814329a5646f0; no voters:
I20260809 13:15:00.887645 13537 leader_election.cc:288] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 13:15:00.887795 13554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:15:00.888422 13554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [term 1 LEADER]: Becoming Leader. State: Replica: 250542ca5b7042dc839814329a5646f0, State: Running, Role: LEADER
I20260809 13:15:00.888948 13554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } }
I20260809 13:15:00.889758 13537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:15:00.891724 13556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 250542ca5b7042dc839814329a5646f0. Latest consensus state: current_term: 1 leader_uuid: "250542ca5b7042dc839814329a5646f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } } }
I20260809 13:15:00.892158 13556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:00.892831 13555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "250542ca5b7042dc839814329a5646f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "250542ca5b7042dc839814329a5646f0" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 46301 } } }
I20260809 13:15:00.893126 13555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:00.894007 13567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:15:00.899082 13567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:15:00.907822 13567 catalog_manager.cc:1347] Generated new cluster ID: d0d16c56820b459dac084cda7b07cbf7
I20260809 13:15:00.907936 13567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:15:00.922163 13567 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:15:00.922654 13567 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:15:00.938301 13567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 250542ca5b7042dc839814329a5646f0: Generated new TSK 0
I20260809 13:15:00.939211 13567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:15:00.956255 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13430
W20260809 13:15:00.957355 13528 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43055
W20260809 13:15:00.958073 10966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13430: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 13:15:00.958199 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13430
2026-08-09T13:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2109 ms)
[----------] 18 tests from AuthTokenIssuingTest (37613 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:15:01 dist-test-slave-42z9 krb5kdc[13585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:15:01 dist-test-slave-42z9 krb5kdc[13585](info): set up 2 sockets
Aug 09 13:15:01 dist-test-slave-42z9 krb5kdc[13585](info): commencing operation
krb5kdc: starting...
W20260809 13:15:02.202978 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:15:02 dist-test-slave-42z9 krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:15:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:15:02.509088 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:44197
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:15:02.872908 13606 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:02.873222 13606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:02.873317 13606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:02.873431 13606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:02.880518 13606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:02.880604 13606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:02.880697 13606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:02.880759 13606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:02.880843 13606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:02.890458 13606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:02.891649 13606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:02.892902 13606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:15:02.904222 13613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:02.904189 13614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:02.904268 13617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:02.904939 13606 server_base.cc:1034] running on GCE node
I20260809 13:15:02.905701 13606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:02.907017 13606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:02.908234 13606 hybrid_clock.cc:648] HybridClock initialized: now 1786281302908172 us; error 32 us; skew 500 ppm
I20260809 13:15:02.908545 13606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:02 dist-test-slave-42z9 krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:02.911545 13606 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:15:02.912818 13606 webserver.cc:460] Webserver started at http://127.10.181.190:34255/ using document root <none> and password file <none>
I20260809 13:15:02.913461 13606 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:02.913563 13606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:02.913883 13606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:02.916656 13606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "6e53d1252c0d41ecaf42ff20a397a681"
format_stamp: "Formatted at 2026-08-09 13:15:02 on dist-test-slave-42z9"
server_key: "bbbfcd05b982b9f280a4da3c98122ebb"
server_key_iv: "e3b8da0d2ebaeee030bd4c13bf7d94e9"
server_key_version: "encryptionkey@0"
I20260809 13:15:02.917357 13606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "6e53d1252c0d41ecaf42ff20a397a681"
format_stamp: "Formatted at 2026-08-09 13:15:02 on dist-test-slave-42z9"
server_key: "bbbfcd05b982b9f280a4da3c98122ebb"
server_key_iv: "e3b8da0d2ebaeee030bd4c13bf7d94e9"
server_key_version: "encryptionkey@0"
I20260809 13:15:02.922112 13606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 13:15:02.925300 13629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:02.926514 13606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 13:15:02.926666 13606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "6e53d1252c0d41ecaf42ff20a397a681"
format_stamp: "Formatted at 2026-08-09 13:15:02 on dist-test-slave-42z9"
server_key: "bbbfcd05b982b9f280a4da3c98122ebb"
server_key_iv: "e3b8da0d2ebaeee030bd4c13bf7d94e9"
server_key_version: "encryptionkey@0"
I20260809 13:15:02.926833 13606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:02.941547 13606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:02.945025 13606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:02.945232 13606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:02.962455 13606 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:44197
I20260809 13:15:02.962472 13725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:44197 every 8 connection(s)
I20260809 13:15:02.963712 13606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:15:02.964890 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13606
I20260809 13:15:02.965106 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:15:02.965956 10966 external_mini_cluster.cc:1428] Setting key 9195e72f93a893d8aa8ef016b2380491
I20260809 13:15:02.968205 13727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:02.976563 13727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:02 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:02.999562 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:02.980043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44197 (local address 127.0.0.1:51982)
0809 13:15:02.980843 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:02.980889 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:02.982802 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:02.983590 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:02.983598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:02.984325 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:02.984765 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:02.984788 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:02.986373 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:02.986378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:02.988015 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:02.988030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:02.989010 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:02.989021 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:02.989229 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:02.989895 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:02.989922 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:02.992382 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:02.995421 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:02.995433 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:02.995445 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:02.995821 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:02.996216 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:02.996221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:02.996224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:02.996368 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:02.997142 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:02.997154 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:02.997353 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 13:15:02.997633 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 13:15:02.998253 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":180,"thread_start_us":341,"threads_started":1}
I20260809 13:15:02.999594 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:02.980044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.10.181.190:44197)
0809 13:15:02.981646 (+ 1602us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:02.981655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:02.981704 (+ 49us) server_negotiation.cc:407] Connection header received
0809 13:15:02.983143 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:02.983149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:02.983355 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:02.983510 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:02.984973 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:02.986220 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:02.988178 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:02.988737 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:02.988889 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:02.992607 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:02.992615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:02.992627 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:02.992655 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:02.995289 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:02.995949 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:02.995955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:02.995957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:02.996103 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:02.996497 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:02.996500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:02.996502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:02.997013 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:02.997204 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:02.997760 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 13:15:02.998125 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1385,"thread_start_us":1154,"threads_started":1}
WARNING: no policy specified for kudu/127.10.181.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.189@KRBTEST.COM" created.
I20260809 13:15:03.002480 13727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:03.003575 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.003144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53938)
0809 13:15:03.003323 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.003443 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 13:15:03.003809 13634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
I20260809 13:15:03.005359 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.004956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53950)
0809 13:15:03.005123 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.005203 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:15:03.005556 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:44841: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
I20260809 13:15:03.038436 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:44841
--webserver_interface=127.10.181.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:15:03.054884 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } attempt: 1
I20260809 13:15:03.056098 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.055636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53962)
0809 13:15:03.055838 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.055950 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:15:03.057204 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.056850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53968)
0809 13:15:03.056988 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.057060 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:15:03.057407 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:44841: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
I20260809 13:15:03.125664 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } attempt: 2
I20260809 13:15:03.126902 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.126424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53972)
0809 13:15:03.126650 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.126748 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:15:03.127905 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.127562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53976)
0809 13:15:03.127711 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.127775 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 13:15:03.128099 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:44841: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
I20260809 13:15:03.219314 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } attempt: 3
I20260809 13:15:03.220521 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.220093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53990)
0809 13:15:03.220290 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.220384 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 13:15:03.221606 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.221217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:53996)
0809 13:15:03.221353 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.221423 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:15:03.221804 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:44841: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
I20260809 13:15:03.365017 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } attempt: 4
I20260809 13:15:03.366230 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.365693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:54006)
0809 13:15:03.365998 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.366098 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260809 13:15:03.367229 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.366877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:54008)
0809 13:15:03.367019 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.367111 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 13:15:03.367406 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:44841: Network error: Client connection negotiation failed: client connection to 127.10.181.189:44841: connect: Connection refused (error 111)
W20260809 13:15:03.377004 13739 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:03.377267 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:03.377339 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:03.377424 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:03.384394 13739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:03.384476 13739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:03.384573 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:03.384634 13739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:03.384717 13739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:03.394093 13739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:44841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.181.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:03.395218 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:03.396459 13739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:15:03.406843 13747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:03.406906 13748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:03.407919 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:03.408468 13739 server_base.cc:1034] running on GCE node
I20260809 13:15:03.409191 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:03.410475 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:03.411692 13739 hybrid_clock.cc:648] HybridClock initialized: now 1786281303411627 us; error 31 us; skew 500 ppm
I20260809 13:15:03.412027 13739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:03 dist-test-slave-42z9 krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:03.415030 13739 init.cc:377] Logged in from keytab as kudu/127.10.181.189@KRBTEST.COM (short username kudu)
I20260809 13:15:03.416308 13739 webserver.cc:460] Webserver started at http://127.10.181.189:41911/ using document root <none> and password file <none>
I20260809 13:15:03.416930 13739 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:03.417011 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:03.417273 13739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:03.420203 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data/instance:
uuid: "3a6c40c5a5c948efbfc531610dfdbe5c"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "5abe710621c43b9130cc1ce9d9736867"
server_key_iv: "8a29c9ffa0e26b9c6880dd05521a253e"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.420902 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal/instance:
uuid: "3a6c40c5a5c948efbfc531610dfdbe5c"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "5abe710621c43b9130cc1ce9d9736867"
server_key_iv: "8a29c9ffa0e26b9c6880dd05521a253e"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.425783 13739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 13:15:03.428975 13764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:03.430248 13739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 13:15:03.430402 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "3a6c40c5a5c948efbfc531610dfdbe5c"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "5abe710621c43b9130cc1ce9d9736867"
server_key_iv: "8a29c9ffa0e26b9c6880dd05521a253e"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.430608 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:03.459631 13739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:03.463238 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:03.463469 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:03.479741 13739 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.189:44841
I20260809 13:15:03.479761 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.189:44841 every 8 connection(s)
I20260809 13:15:03.480954 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
I20260809 13:15:03.483086 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13739
I20260809 13:15:03.483232 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-1/wal/instance
I20260809 13:15:03.483649 10966 external_mini_cluster.cc:1428] Setting key 70945b2c0bee11bb1ae636c3f359424d
I20260809 13:15:03.484647 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:03.491144 13844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:03 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:03.510185 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.493657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.10.181.190:44197)
0809 13:15:03.493863 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:03.493869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:03.494890 (+ 1021us) server_negotiation.cc:407] Connection header received
0809 13:15:03.496699 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:03.496705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:03.496785 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:03.496901 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:03.498328 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.499257 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.500756 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.501183 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.501299 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.504492 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:03.504497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:03.504500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:03.504531 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:03.507204 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.507977 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.507983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.507987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.508111 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.508550 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.508555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.508559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.508834 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:03.508975 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:03.509637 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 13:15:03.509861 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:15:03.511013 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.493993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44197 (local address 127.0.0.1:51992)
0809 13:15:03.494717 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.494769 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:03.496422 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:03.497060 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:03.497067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:03.497619 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:03.498157 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.498174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.499443 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.499448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.500612 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.500625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.501410 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.501417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.501654 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.502246 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:03.502267 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:03.504289 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:03.507350 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.507359 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.507392 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.507842 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.508215 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.508220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.508224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.508400 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.508966 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:03.508984 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:03.509274 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 13:15:03.509519 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 13:15:03.509976 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":292,"threads_started":1}
I20260809 13:15:03.512602 13844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.181.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.188@KRBTEST.COM" created.
Aug 09 13:15:03 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:03.525950 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.513338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:54024)
0809 13:15:03.513543 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.513560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:03.513663 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:03.514321 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:03.514326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:03.514730 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:03.514848 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.514856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.516177 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.516183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.517025 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.517037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.517863 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.517869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.518043 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.518573 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:03.518590 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:03.520686 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:03.523570 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.523575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.523578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.524022 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.524348 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.524352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.524355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.524537 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.525059 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:03.525063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:03.525202 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 13:15:03.525339 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 13:15:03.525619 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 13:15:03.526014 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.513391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.10.181.189:44841)
0809 13:15:03.513959 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:03.513964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:03.513988 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:15:03.514025 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:03.514030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:03.514181 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:03.514320 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:03.515007 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.516049 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.517147 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.517732 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.517841 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.520907 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:03.520918 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:03.520930 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:03.520957 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:03.523442 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.524137 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.524143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.524146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.524252 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.524637 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.524640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.524642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.524946 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:03.525109 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:03.525474 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 13:15:03.525715 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":430,"thread_start_us":269,"threads_started":1}
I20260809 13:15:03.528402 13844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:03.529373 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.528915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45460)
0809 13:15:03.529071 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.529222 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 13:15:03.529670 13769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.531180 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.530854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45474)
0809 13:15:03.531006 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.531088 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 13:15:03.531395 13844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
I20260809 13:15:03.556416 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:43219
--webserver_interface=127.10.181.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:15:03.580693 13844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 1
I20260809 13:15:03.581810 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.581331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45480)
0809 13:15:03.581551 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.581655 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:15:03.582878 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.582539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45492)
0809 13:15:03.582694 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.582769 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 13:15:03.583096 13844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.651297 13844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 2
I20260809 13:15:03.652488 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.652032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45494)
0809 13:15:03.652236 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.652326 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 13:15:03.653510 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.653161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45502)
0809 13:15:03.653293 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.653361 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 13:15:03.653707 13844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.666604 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } attempt: 5
Aug 09 13:15:03 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:03.683565 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.667247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.10.181.189:44841)
0809 13:15:03.667436 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:03.667442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:03.667462 (+ 20us) server_negotiation.cc:407] Connection header received
0809 13:15:03.667609 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:03.667613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:03.667688 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:03.667770 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:03.668588 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.669306 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.670499 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.670942 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.671053 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.678673 (+ 7620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:03.678678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:03.678681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:03.678714 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:03.681093 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.681899 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.681903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.681906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.682026 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:03.682390 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:03.682393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:03.682395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:03.682670 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:03.682778 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:03.683204 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 13:15:03.683363 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 13:15:03.683736 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.667163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:54026)
0809 13:15:03.667324 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.667343 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:03.667473 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:03.667830 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:03.667837 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:03.668235 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:03.668434 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.668444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.669471 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.669477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.670346 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:03.670360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:03.671092 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:03.671098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:03.671307 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:03.676063 (+ 4756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:03.676092 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:03.678473 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:03.681236 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.681241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.681245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.681793 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.682125 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:03.682129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:03.682133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:03.682300 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:03.682776 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:03.682780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:03.682963 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 13:15:03.683110 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 13:15:03.683400 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 13:15:03.684736 13727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:03.685715 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.685258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45506)
0809 13:15:03.685459 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.685555 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 13:15:03.686553 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.686272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45508)
0809 13:15:03.686381 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.686454 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 13:15:03.686749 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.737951 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 1
I20260809 13:15:03.739104 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.738649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45520)
0809 13:15:03.738840 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.738934 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 13:15:03.740051 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.739765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45526)
0809 13:15:03.739882 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.739950 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 13:15:03.740265 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.744889 13844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 3
I20260809 13:15:03.745814 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.745419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45528)
0809 13:15:03.745600 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.745675 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 13:15:03.746781 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.746371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45542)
0809 13:15:03.746591 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.746665 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260809 13:15:03.746978 13844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.808446 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 2
I20260809 13:15:03.809675 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.809157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45550)
0809 13:15:03.809387 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.809515 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 13:15:03.810580 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.810258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45558)
0809 13:15:03.810397 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.810466 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 13:15:03.810783 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
I20260809 13:15:03.890210 13844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 4
I20260809 13:15:03.891476 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.891015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45568)
0809 13:15:03.891227 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.891318 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:15:03.892400 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.892089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45576)
0809 13:15:03.892230 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.892298 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 13:15:03.892598 13844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
W20260809 13:15:03.897934 13854 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:03.898252 13854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:03.898343 13854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:03.898459 13854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:03.905788 13854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:03.905874 13854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:03.905967 13854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:03.906033 13854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:03.906122 13854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:03.915649 13854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:44197,127.10.181.189:44841,127.10.181.188:43219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:43219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.181.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:03.916879 13854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:03.917006 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 3
I20260809 13:15:03.918090 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.917658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45590)
0809 13:15:03.917833 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.917927 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 13:15:03.918457 13854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 13:15:03.919085 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.918716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45602)
0809 13:15:03.918894 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:03.918981 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 13:15:03.919270 13727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:43219: Network error: Client connection negotiation failed: client connection to 127.10.181.188:43219: connect: Connection refused (error 111)
W20260809 13:15:03.930161 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:03.930207 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:03.930388 13867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:03.930972 13854 server_base.cc:1034] running on GCE node
I20260809 13:15:03.931699 13854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:03.932900 13854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:03.934111 13854 hybrid_clock.cc:648] HybridClock initialized: now 1786281303934039 us; error 41 us; skew 500 ppm
I20260809 13:15:03.934470 13854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:03 dist-test-slave-42z9 krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:03.937417 13854 init.cc:377] Logged in from keytab as kudu/127.10.181.188@KRBTEST.COM (short username kudu)
I20260809 13:15:03.938804 13854 webserver.cc:460] Webserver started at http://127.10.181.188:40219/ using document root <none> and password file <none>
I20260809 13:15:03.939451 13854 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:03.939539 13854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:03.939800 13854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:03.942613 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data/instance:
uuid: "26bad0466b704697bfda5408668ff401"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "fc84ba2579fba464a381c547238c4844"
server_key_iv: "7827651ac04f48f957205ccf81fee0e0"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.943320 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal/instance:
uuid: "26bad0466b704697bfda5408668ff401"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "fc84ba2579fba464a381c547238c4844"
server_key_iv: "7827651ac04f48f957205ccf81fee0e0"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.948076 13854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 13:15:03.951325 13884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:03.952672 13854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 13:15:03.952833 13854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "26bad0466b704697bfda5408668ff401"
format_stamp: "Formatted at 2026-08-09 13:15:03 on dist-test-slave-42z9"
server_key: "fc84ba2579fba464a381c547238c4844"
server_key_iv: "7827651ac04f48f957205ccf81fee0e0"
server_key_version: "encryptionkey@0"
I20260809 13:15:03.953029 13854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:03.966782 13854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:03.970600 13854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:03.970834 13854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:03.987860 13854 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.188:43219
I20260809 13:15:03.987879 13956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.188:43219 every 8 connection(s)
I20260809 13:15:03.989068 13854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
I20260809 13:15:03.992515 13957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:03.995434 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 13854
I20260809 13:15:03.995599 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786280946828509-10966-0/minicluster-data/master-2/wal/instance
I20260809 13:15:03.996016 10966 external_mini_cluster.cc:1428] Setting key d6ae900f53d18e4e89abef6d09a6626e
I20260809 13:15:03.999454 13957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:04.016772 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.002659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.10.181.190:44197)
0809 13:15:04.002849 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.002855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.003551 (+ 696us) server_negotiation.cc:407] Connection header received
0809 13:15:04.003733 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.003739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.003818 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.003911 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.004999 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.005754 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.007177 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.007607 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.007718 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.011241 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.011248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.011253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.011292 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.014052 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.014816 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.014820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.014822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.014959 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.015339 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.015344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.015347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.015699 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.015839 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.016334 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.016512 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 13:15:04.017625 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.002597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:44197 (local address 127.0.0.1:52004)
0809 13:15:04.003376 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:04.003415 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:04.003593 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:04.003981 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:04.003986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:04.004588 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:04.004843 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.004852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.005915 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.005919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.007035 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.007050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.007821 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.007829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.007980 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.008497 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:04.008519 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:04.011020 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:04.014165 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.014175 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.014182 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.014705 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.015076 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.015080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.015083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.015241 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.015832 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:04.015840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:04.016038 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 13:15:04.016249 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 13:15:04.016526 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":422,"threads_started":1}
I20260809 13:15:04.017596 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:03.997750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.10.181.188:43219)
0809 13:15:03.998510 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:03.998520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:03.998561 (+ 41us) server_negotiation.cc:407] Connection header received
0809 13:15:04.000142 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.000163 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.000528 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.000832 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.002588 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.004265 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.005509 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.006145 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.006323 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.010358 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.010369 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.010383 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.010420 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.013180 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.013971 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.013978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.013984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.014161 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.014519 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.014524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.014528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.015106 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.015406 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.015802 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.016249 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":350,"threads_started":1}
I20260809 13:15:04.019275 13957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:04.032992 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.019989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:44841 (local address 127.0.0.1:54032)
0809 13:15:04.020164 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:04.020179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:04.020250 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:04.020564 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:04.020569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:04.020996 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:04.021134 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.021143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.022463 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.022467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.023299 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.023310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.023995 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.024001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.024175 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.025291 (+ 1116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:04.025306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:04.027358 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:04.030710 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.030716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.030720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.031079 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.031453 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.031458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.031462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.031611 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.032200 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:04.032204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:04.032325 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 13:15:04.032482 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 13:15:04.032744 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 13:15:04.033260 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.019981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.10.181.189:44841)
0809 13:15:04.020183 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.020190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.020213 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:15:04.020353 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.020359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.020424 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.020523 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.021277 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.022325 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.023441 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.023864 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.023973 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.027562 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.027567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.027570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.027603 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.030585 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.031197 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.031203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.031207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.031346 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.031714 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.031719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.031723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.032065 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.032225 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.032571 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.032753 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:15:04.034207 13957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:04.035079 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.020904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.10.181.190:44197)
0809 13:15:04.021046 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.021050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.021063 (+ 13us) server_negotiation.cc:407] Connection header received
0809 13:15:04.021097 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.021100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.021154 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.021227 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.022712 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.023412 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.024632 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.025037 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.025127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.029117 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.029123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.029126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.029156 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.031719 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.032426 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.032430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.032434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.032583 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.032989 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.032994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.032997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.033294 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.034382 (+ 1088us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.034567 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.034774 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:04.047005 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.034920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45614)
0809 13:15:04.035065 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:04.035080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:04.035158 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:04.035863 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:04.035866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:04.036249 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:04.036376 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.036384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.037510 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.037515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.038430 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.038445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.039111 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.039118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.039280 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.039802 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:04.039817 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:04.041940 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:04.044785 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.044788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.044791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.045120 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.045486 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.045489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.045492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.045628 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.046098 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:04.046103 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:04.046214 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 13:15:04.046339 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 13:15:04.046599 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 13:15:04.047050 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.035114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.10.181.188:43219)
0809 13:15:04.035606 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.035611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.035629 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:15:04.035676 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.035680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.035747 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.035843 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.036507 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.037372 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.038565 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.038967 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.039064 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.042129 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.042134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.042137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.042165 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.044657 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.045232 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.045237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.045240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.045352 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.045733 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.045737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.045741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.045986 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.046112 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.046446 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.046766 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":375}
I20260809 13:15:04.049952 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.036619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54038 (local address 127.10.181.189:44841)
0809 13:15:04.036775 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.036780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.036796 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:15:04.036888 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.036893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.036965 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.037060 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.038189 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.038998 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.040346 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.040935 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.041026 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.045097 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.045102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.045105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.045134 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.047602 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.048256 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.048260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.048262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.048412 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.048796 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.048799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.048801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.049028 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.049224 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.049554 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.049718 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:15:04.052496 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: Bootstrap starting.
I20260809 13:15:04.054510 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:15:04.056010 13957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:04.057197 13957 log.cc:824] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:04.060401 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: No bootstrap required, opened a new log
I20260809 13:15:04.057240 13686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6e53d1252c0d41ecaf42ff20a397a681"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52016
I20260809 13:15:04.071017 13957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.071401 13957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bad0466b704697bfda5408668ff401, State: Initialized, Role: FOLLOWER
I20260809 13:15:04.071900 13957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.072755 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.073067 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.073845 13957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 13:15:04.082518 13987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:15:04.082650 13987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:15:04.080829 13854 master_runner.cc:186] Error getting master registration for 127.10.181.190:44197: OK, instance_id { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" instance_seqno: 1786281302957290 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:15:04.091429 13854 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:15:04.096481 13727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 4
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
I20260809 13:15:04.111176 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.097149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45624)
0809 13:15:04.097330 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:04.097349 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:04.097501 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:04.097850 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:04.097855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:04.098299 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:04.098481 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.098491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.099553 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.099559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.100659 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.100676 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.101540 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.101546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.101755 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.102367 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:04.102390 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:04.104830 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:04.108660 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.108666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.108670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.109044 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.109465 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.109470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.109474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.109625 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.110186 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:04.110191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:04.110367 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 13:15:04.110480 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 13:15:04.110772 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:15:04.111212 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.097315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45624 (local address 127.10.181.188:43219)
0809 13:15:04.097548 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.097553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.097577 (+ 24us) server_negotiation.cc:407] Connection header received
0809 13:15:04.097645 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.097650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.097715 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.097795 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.098656 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.099392 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.100816 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.101343 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.101479 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.105013 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.105018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.105021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.105051 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.108540 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.109168 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.109174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.109178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.109339 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.109734 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.109740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.109743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.110067 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.110239 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.110640 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.110853 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 13:15:04.118288 13727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: Bootstrap starting.
I20260809 13:15:04.122572 13727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:04.123981 13727 log.cc:824] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:04.127511 13727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: No bootstrap required, opened a new log
I20260809 13:15:04.135143 13727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.135572 13727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e53d1252c0d41ecaf42ff20a397a681, State: Initialized, Role: FOLLOWER
I20260809 13:15:04.136092 13727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.137143 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.137529 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.138505 13727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:15:04.138578 13686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 13:15:04.138777 13686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:04.141511 13686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.142927 13686 leader_election.cc:288] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [CANDIDATE]: Term 1 election: Requested vote from peers 3a6c40c5a5c948efbfc531610dfdbe5c (127.10.181.189:44841), 26bad0466b704697bfda5408668ff401 (127.10.181.188:43219)
I20260809 13:15:04.143608 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e53d1252c0d41ecaf42ff20a397a681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26bad0466b704697bfda5408668ff401"
I20260809 13:15:04.144138 13926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:04.147763 13926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e53d1252c0d41ecaf42ff20a397a681 in term 1.
I20260809 13:15:04.148506 13635 leader_election.cc:302] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bad0466b704697bfda5408668ff401, 6e53d1252c0d41ecaf42ff20a397a681; no voters:
I20260809 13:15:04.148967 13994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:15:04.164512 13994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [term 1 LEADER]: Becoming Leader. State: Replica: 6e53d1252c0d41ecaf42ff20a397a681, State: Running, Role: LEADER
I20260809 13:15:04.165167 13994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.173249 14008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e53d1252c0d41ecaf42ff20a397a681. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.173702 14008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:04.180709 14020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:15:04.188737 14020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:15:04.191795 13844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } attempt: 5
W20260809 13:15:04.197660 13638 tablet.cc:2369] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 13:15:04 dist-test-slave-42z9 krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
I20260809 13:15:04.215134 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.192714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.10.181.188:43219)
0809 13:15:04.192908 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.192914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.192930 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:15:04.193003 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.193008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.193093 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.193191 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.194189 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.195359 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.199341 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.200087 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.200219 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.206226 (+ 6007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.206233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.206239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.206281 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.210254 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.211484 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.211503 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.211507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.211673 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.212416 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.212426 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.212430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.213035 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.213283 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.214131 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.214543 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 13:15:04.216729 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.192337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:43219 (local address 127.0.0.1:45640)
0809 13:15:04.192722 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:04.192738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:04.192833 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:04.193222 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:04.193227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:04.193777 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:04.194000 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.194012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.195547 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.195553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.199165 (+ 3612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:04.199181 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.200467 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.200474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:04.200719 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.202637 (+ 1918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:04.202662 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:04.205937 (+ 3275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:04.210507 (+ 4570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.210513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.210517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.211266 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.211906 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:04.211911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:04.211915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:04.212236 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:04.213405 (+ 1169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:04.213410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:04.213665 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 13:15:04.215738 (+ 2073us) client_negotiation.cc:240] Negotiation successful
0809 13:15:04.216182 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":2107}
I20260809 13:15:04.219806 13926 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [term 1 FOLLOWER]: Refusing update from remote peer 6e53d1252c0d41ecaf42ff20a397a681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:15:04.221254 14008 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 13:15:04.222689 13891 tablet.cc:2369] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 13:15:04.223866 13844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: Bootstrap starting.
I20260809 13:15:04.218410 13606 master_runner.cc:186] Error getting master registration for 127.10.181.189:44841: OK, instance_id { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" instance_seqno: 1786281303475197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:15:04.228075 13606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:15:04.227815 13844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:04.229452 13844 log.cc:824] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:04.243212 13844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: No bootstrap required, opened a new log
I20260809 13:15:04.252636 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e53d1252c0d41ecaf42ff20a397a681. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.252956 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.254906 14035 mvcc.cc:205] Tried to move back new op lower bound from 7316608222071631872 to 7316608221867003904. Current Snapshot: MvccSnapshot[applied={T|T < 7316608222071631872}]
I20260809 13:15:04.255970 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.256214 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.262426 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.175639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.10.181.190:44197)
0809 13:15:04.175820 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.175826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.188145 (+ 12319us) server_negotiation.cc:407] Connection header received
0809 13:15:04.188832 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.188838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.188926 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.189014 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.201152 (+ 12138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.202375 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.226145 (+ 23770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.226908 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.227037 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.249952 (+ 22915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.249969 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.249974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.250017 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.253724 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.258053 (+ 4329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.258060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.258064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.258248 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.259356 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.259362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.259366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.259807 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.260066 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.261780 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.262049 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:15:04.262650 13844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.263010 13844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a6c40c5a5c948efbfc531610dfdbe5c, State: Initialized, Role: FOLLOWER
I20260809 13:15:04.263499 13844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } }
I20260809 13:15:04.263967 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.173566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.10.181.189:44841)
0809 13:15:04.173817 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.173824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.188009 (+ 14185us) server_negotiation.cc:407] Connection header received
0809 13:15:04.189565 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.189572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.189658 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.189727 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.196158 (+ 6431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.197298 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.227289 (+ 29991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.227969 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.228080 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.245604 (+ 17524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.245611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.245616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.245656 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.249669 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.258698 (+ 9029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.258705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.258709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.258888 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.260537 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.260542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.260549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.260930 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.261196 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.263268 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.263512 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":96}
I20260809 13:15:04.264747 14046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.265480 14046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.270334 14020 catalog_manager.cc:1347] Generated new cluster ID: aff73e1bdbe04a99a4519ebb8ec167b8
I20260809 13:15:04.270433 14020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:15:04.270885 14008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.271149 14008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:04.273645 14040 mvcc.cc:205] Tried to move back new op lower bound from 7316608222071631872 to 7316608221867003904. Current Snapshot: MvccSnapshot[applied={T|T < 7316608222071631872}]
I20260809 13:15:04.279094 13844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:15:04.279178 13804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e53d1252c0d41ecaf42ff20a397a681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c"
I20260809 13:15:04.279649 13804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:04.281950 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:04.218013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.10.181.188:43219)
0809 13:15:04.228275 (+ 10262us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.228280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.232178 (+ 3898us) server_negotiation.cc:407] Connection header received
0809 13:15:04.233807 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.233813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.233895 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.233979 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.243928 (+ 9949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.245015 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.267294 (+ 22279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:04.268050 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:04.268149 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:04.274032 (+ 5883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:04.274039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:04.274044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:04.274101 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:04.277835 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.278697 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.278703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.278706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.278854 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:04.280060 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:04.280065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:04.280069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:04.280417 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:04.280657 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:04.281219 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 13:15:04.281559 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10138}
I20260809 13:15:04.282994 13804 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e53d1252c0d41ecaf42ff20a397a681 in term 1.
I20260809 13:15:04.284065 13803 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [term 1 FOLLOWER]: Refusing update from remote peer 6e53d1252c0d41ecaf42ff20a397a681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:15:04.284933 13994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:15:04.301318 14020 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:15:04.303036 14020 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:15:04.322902 14020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681: Generated new TSK 0
I20260809 13:15:04.323858 14020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:15:04.334942 14046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e53d1252c0d41ecaf42ff20a397a681. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.334916 14063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.335327 14046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.335361 14063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:04.335487 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e53d1252c0d41ecaf42ff20a397a681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e53d1252c0d41ecaf42ff20a397a681" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 44197 } } peers { permanent_uuid: "3a6c40c5a5c948efbfc531610dfdbe5c" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 44841 } } peers { permanent_uuid: "26bad0466b704697bfda5408668ff401" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 43219 } } }
I20260809 13:15:04.335784 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e53d1252c0d41ecaf42ff20a397a681 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:04.350181 14070 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 13:15:04.423769 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.414258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.10.181.189:44841)
0809 13:15:04.414432 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.414438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.420927 (+ 6489us) server_negotiation.cc:407] Connection header received
0809 13:15:04.422165 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.422171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.422257 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.422329 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.423530 (+ 1201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54054: BlockingRecv error: recv got EOF from 127.0.0.1:54054 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 13:15:04.427770 13961 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.421487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.10.181.188:43219)
0809 13:15:04.421688 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.421694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.425178 (+ 3484us) server_negotiation.cc:407] Connection header received
0809 13:15:04.425885 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.425891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.425969 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.426047 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.427533 (+ 1486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45644: BlockingRecv error: recv got EOF from 127.0.0.1:45644 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 13:15:04.427953 13731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.413829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.10.181.190:44197)
0809 13:15:04.414042 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.414048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.425093 (+ 11045us) server_negotiation.cc:407] Connection header received
0809 13:15:04.425710 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.425717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.425798 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.425866 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.427716 (+ 1850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52044: BlockingRecv error: recv got EOF from 127.0.0.1:52044 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 13:15:04.434835 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.432730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54058 (local address 127.10.181.189:44841)
0809 13:15:04.432885 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.432891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.433350 (+ 459us) server_negotiation.cc:407] Connection header received
0809 13:15:04.433585 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.433591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.433714 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.433822 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.434607 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54058: BlockingRecv error: recv got EOF from 127.0.0.1:54058 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 13:15:04.447266 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.440146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.10.181.189:44841)
0809 13:15:04.440285 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.440290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.445961 (+ 5671us) server_negotiation.cc:407] Connection header received
0809 13:15:04.446131 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.446137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.446220 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.446322 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.447066 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54066: BlockingRecv error: recv got EOF from 127.0.0.1:54066 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 13:15:04.457664 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.455457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.10.181.189:44841)
0809 13:15:04.455639 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.455645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.456149 (+ 504us) server_negotiation.cc:407] Connection header received
0809 13:15:04.456431 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.456437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.456530 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.456623 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.457398 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54080: BlockingRecv error: recv got EOF from 127.0.0.1:54080 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 13:15:04.473481 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.471398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.10.181.189:44841)
0809 13:15:04.471623 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.471629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.472099 (+ 470us) server_negotiation.cc:407] Connection header received
0809 13:15:04.472276 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.472281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.472368 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.472489 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.473211 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54092: BlockingRecv error: recv got EOF from 127.0.0.1:54092 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260809 13:15:04.511780 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.509339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54104 (local address 127.10.181.189:44841)
0809 13:15:04.509555 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.509561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.510125 (+ 564us) server_negotiation.cc:407] Connection header received
0809 13:15:04.510420 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.510426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.510529 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.510603 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.511553 (+ 950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54104: BlockingRecv error: recv got EOF from 127.0.0.1:54104 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 13:15:04.537887 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.535455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.10.181.189:44841)
0809 13:15:04.535639 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.535646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.536153 (+ 507us) server_negotiation.cc:407] Connection header received
0809 13:15:04.536451 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.536470 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.536567 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.536649 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.537657 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54112: BlockingRecv error: recv got EOF from 127.0.0.1:54112 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 13:15:04.577138 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.574441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.10.181.189:44841)
0809 13:15:04.574669 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.574675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.575207 (+ 532us) server_negotiation.cc:407] Connection header received
0809 13:15:04.575586 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.575592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.575688 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.575761 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.576881 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54118: BlockingRecv error: recv got EOF from 127.0.0.1:54118 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260809 13:15:04.647692 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.645700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.10.181.189:44841)
0809 13:15:04.645877 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.645883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.646393 (+ 510us) server_negotiation.cc:407] Connection header received
0809 13:15:04.646576 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.646580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.646662 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.646748 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.647513 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54120: BlockingRecv error: recv got EOF from 127.0.0.1:54120 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 13:15:04.781153 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:04.779308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.10.181.189:44841)
0809 13:15:04.779527 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:04.779534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:04.779873 (+ 339us) server_negotiation.cc:407] Connection header received
0809 13:15:04.780055 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:04.780059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:04.780154 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:04.780241 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:04.780906 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54122: BlockingRecv error: recv got EOF from 127.0.0.1:54122 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 13:15:05.042832 13851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:05.040974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.10.181.189:44841)
0809 13:15:05.041199 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.041205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.041361 (+ 156us) server_negotiation.cc:407] Connection header received
0809 13:15:05.041615 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.041634 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.041711 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.041797 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:05.042576 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54134: BlockingRecv error: recv got EOF from 127.0.0.1:54134 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 13:15:05.084753 13987 catalog_manager.cc:1259] Loaded cluster ID: aff73e1bdbe04a99a4519ebb8ec167b8
I20260809 13:15:05.084858 13987 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: loading cluster ID for follower catalog manager: success
I20260809 13:15:05.087460 13987 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: acquiring CA information for follower catalog manager: success
I20260809 13:15:05.089229 13987 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26bad0466b704697bfda5408668ff401: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:15:05.355350 14070 catalog_manager.cc:1259] Loaded cluster ID: aff73e1bdbe04a99a4519ebb8ec167b8
I20260809 13:15:05.355497 14070 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: loading cluster ID for follower catalog manager: success
I20260809 13:15:05.359277 14070 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: acquiring CA information for follower catalog manager: success
I20260809 13:15:05.361717 14070 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a6c40c5a5c948efbfc531610dfdbe5c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:15:05.568698 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:05.559271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.10.181.189:44841)
0809 13:15:05.561986 (+ 2715us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.561992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.562367 (+ 375us) server_negotiation.cc:407] Connection header received
0809 13:15:05.562618 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.562623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.562784 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.562907 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:05.563413 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.564290 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.566262 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.566701 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.566794 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:05.567422 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:05.567988 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:05.568100 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:05.568252 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 13:15:05.568324 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2546,"thread_start_us":2382,"threads_started":1}
I20260809 13:15:05.581895 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:05.575331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.10.181.188:43219)
0809 13:15:05.576030 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.576034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.576050 (+ 16us) server_negotiation.cc:407] Connection header received
0809 13:15:05.576142 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.576145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.576284 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.576418 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:05.576884 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.577662 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.579442 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.579856 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.579942 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:05.580589 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:05.581238 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:05.581417 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:05.581505 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 13:15:05.581592 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":461,"threads_started":1}
I20260809 13:15:05.609097 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:05.589579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.10.181.189:44841)
0809 13:15:05.589738 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.589744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.602187 (+ 12443us) server_negotiation.cc:407] Connection header received
0809 13:15:05.602571 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.602577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.602687 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.602754 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:05.603525 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.604798 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.606918 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.607580 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.607683 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:05.608283 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:05.608405 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:05.608469 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:05.608702 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 13:15:05.608783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 13:15:05.612574 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:05.596576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.10.181.188:43219)
0809 13:15:05.596749 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.596754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.606480 (+ 9726us) server_negotiation.cc:407] Connection header received
0809 13:15:05.606665 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.606669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.606770 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.606865 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:05.607737 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.608715 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.610703 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.611265 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.611359 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:05.611827 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:05.611951 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:05.612013 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:05.612216 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 13:15:05.612295 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 13:15:05.614084 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:05.589104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52054 (local address 127.10.181.190:44197)
0809 13:15:05.605503 (+ 16399us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:05.605509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:05.605532 (+ 23us) server_negotiation.cc:407] Connection header received
0809 13:15:05.605661 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:05.605667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:05.605881 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:05.607600 (+ 1719us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:05.607785 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.608851 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.611003 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:05.611642 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:05.611778 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:05.612294 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:05.612749 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:05.612865 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:05.613699 (+ 834us) server_negotiation.cc:299] Negotiation successful
0809 13:15:05.613791 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16233,"thread_start_us":17533,"threads_started":1}
I20260809 13:15:05.618643 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13606
I20260809 13:15:05.619719 13720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:05.735826 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13606
I20260809 13:15:05.763437 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13739
I20260809 13:15:05.764585 13833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:05.868155 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13739
I20260809 13:15:05.896623 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 13854
I20260809 13:15:05.897751 13952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:06.002420 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 13854
2026-08-09T13:15:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5054 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 13:15:06 dist-test-slave-42z9 krb5kdc[14209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 13:15:06 dist-test-slave-42z9 krb5kdc[14209](info): set up 2 sockets
Aug 09 13:15:06 dist-test-slave-42z9 krb5kdc[14209](info): commencing operation
krb5kdc: starting...
W20260809 13:15:07.254278 10966 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/test-user.keytab.
Aug 09 13:15:07 dist-test-slave-42z9 krb5kdc[14209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T13:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T13:15:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.181.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
Entry for principal kudu/127.10.181.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab.
I20260809 13:15:07.557348 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39001
--webserver_interface=127.10.181.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 13:15:07.886423 14229 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:07.886722 14229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:07.886794 14229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:07.886874 14229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:07.894107 14229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:07.894191 14229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:07.894255 14229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:07.894306 14229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:07.894366 14229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:07.904039 14229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.190.keytab
--principal=kudu/127.10.181.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.190:39001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.181.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:07.905192 14229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:07.906523 14229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:15:07.918004 14244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:07.918026 14239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:07.918205 14238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:07.918815 14229 server_base.cc:1034] running on GCE node
I20260809 13:15:07.919590 14229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:07.921016 14229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:07.922232 14229 hybrid_clock.cc:648] HybridClock initialized: now 1786281307922161 us; error 35 us; skew 500 ppm
I20260809 13:15:07.922549 14229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:07 dist-test-slave-42z9 krb5kdc[14209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:07.925578 14229 init.cc:377] Logged in from keytab as kudu/127.10.181.190@KRBTEST.COM (short username kudu)
I20260809 13:15:07.926908 14229 webserver.cc:460] Webserver started at http://127.10.181.190:36853/ using document root <none> and password file <none>
I20260809 13:15:07.927580 14229 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:07.927695 14229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:07.928007 14229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:07.930938 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data/instance:
uuid: "74ce6d8219f645acb66cf5c529005721"
format_stamp: "Formatted at 2026-08-09 13:15:07 on dist-test-slave-42z9"
server_key: "a82cd331336c7eb0f11402c7fbbee41f"
server_key_iv: "4a8006f710376c027dc3bf279d56d475"
server_key_version: "encryptionkey@0"
I20260809 13:15:07.931725 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance:
uuid: "74ce6d8219f645acb66cf5c529005721"
format_stamp: "Formatted at 2026-08-09 13:15:07 on dist-test-slave-42z9"
server_key: "a82cd331336c7eb0f11402c7fbbee41f"
server_key_iv: "4a8006f710376c027dc3bf279d56d475"
server_key_version: "encryptionkey@0"
I20260809 13:15:07.936672 14229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 13:15:07.939987 14252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:07.941171 14229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 13:15:07.941330 14229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal
uuid: "74ce6d8219f645acb66cf5c529005721"
format_stamp: "Formatted at 2026-08-09 13:15:07 on dist-test-slave-42z9"
server_key: "a82cd331336c7eb0f11402c7fbbee41f"
server_key_iv: "4a8006f710376c027dc3bf279d56d475"
server_key_version: "encryptionkey@0"
I20260809 13:15:07.941581 14229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:07.963384 14229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:07.967599 14229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:07.967854 14229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:07.985026 14229 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.190:39001
I20260809 13:15:07.985033 14334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.190:39001 every 8 connection(s)
I20260809 13:15:07.986397 14229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/data/info.pb
I20260809 13:15:07.989971 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:07.993134 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14229
I20260809 13:15:07.993295 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-0/wal/instance
I20260809 13:15:07.993733 10966 external_mini_cluster.cc:1428] Setting key 8206f91b1946549adb3e28edd194ce35
I20260809 13:15:07.996444 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:08.017838 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:07.999345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39001 (local address 127.0.0.1:35046)
0809 13:15:08.000143 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.000192 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:08.002095 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:08.002858 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:08.002865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:08.003548 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:08.003878 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.003895 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.005407 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.005411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.006466 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.006478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.007361 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.007368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.007531 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.008113 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:08.008136 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:08.010400 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:08.014122 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.014133 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.014141 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.014526 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.014980 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.014983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.014985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.015134 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.016020 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:08.016025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:08.016222 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 13:15:08.016416 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 13:15:08.016785 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":319,"threads_started":1}
I20260809 13:15:08.017879 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:07.999346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.10.181.190:39001)
0809 13:15:08.000904 (+ 1558us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:08.000912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:08.000964 (+ 52us) server_negotiation.cc:407] Connection header received
0809 13:15:08.002424 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:08.002430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:08.002653 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:08.002816 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:08.004070 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.005260 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.006604 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.007136 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.007323 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.010640 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:08.010650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:08.010665 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:08.010701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:08.013991 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.014673 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.014681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.014684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.014876 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.015260 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.015266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.015269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.015906 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:08.016095 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:08.016586 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 13:15:08.016990 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":1357,"thread_start_us":1124,"threads_started":1}
I20260809 13:15:08.020560 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:08.021559 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.021161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51246)
0809 13:15:08.021318 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.021418 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 13:15:08.021803 14258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
I20260809 13:15:08.023608 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.023053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51250)
0809 13:15:08.023419 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.023497 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278}
W20260809 13:15:08.023815 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:32953: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.181.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
Entry for principal kudu/127.10.181.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab.
I20260809 13:15:08.066953 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:32953
--webserver_interface=127.10.181.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:15:08.073148 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } attempt: 1
I20260809 13:15:08.074460 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.073920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51258)
0809 13:15:08.074150 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.074278 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 13:15:08.075454 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.075058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51272)
0809 13:15:08.075227 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.075307 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 13:15:08.075649 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:32953: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
I20260809 13:15:08.143878 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } attempt: 2
I20260809 13:15:08.145051 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.144534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51282)
0809 13:15:08.144807 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.144908 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 13:15:08.146169 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.145799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51292)
0809 13:15:08.145947 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.146028 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 13:15:08.146344 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:32953: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
I20260809 13:15:08.237581 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } attempt: 3
I20260809 13:15:08.238687 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.238244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51304)
0809 13:15:08.238448 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.238540 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 13:15:08.239712 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.239344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51306)
0809 13:15:08.239499 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.239565 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 13:15:08.239910 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:32953: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
I20260809 13:15:08.383172 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } attempt: 4
I20260809 13:15:08.384441 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.383906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51318)
0809 13:15:08.384198 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.384304 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 13:15:08.385411 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.385066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51330)
0809 13:15:08.385216 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.385283 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 13:15:08.385609 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.189:32953: Network error: Client connection negotiation failed: client connection to 127.10.181.189:32953: connect: Connection refused (error 111)
W20260809 13:15:08.402488 14346 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:08.402802 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:08.402895 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:08.403005 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 13:15:08.410115 14346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:08.410207 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:08.410295 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:08.410362 14346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:08.410449 14346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:08.420205 14346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.189.keytab
--principal=kudu/127.10.181.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.189:32953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.181.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:08.421506 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:08.422820 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 13:15:08.433843 14355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:08.433921 14356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:08.433941 14359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:08.435002 14346 server_base.cc:1034] running on GCE node
I20260809 13:15:08.435717 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:08.437053 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:08.438278 14346 hybrid_clock.cc:648] HybridClock initialized: now 1786281308438207 us; error 42 us; skew 500 ppm
I20260809 13:15:08.438591 14346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:08.441591 14346 init.cc:377] Logged in from keytab as kudu/127.10.181.189@KRBTEST.COM (short username kudu)
I20260809 13:15:08.442951 14346 webserver.cc:460] Webserver started at http://127.10.181.189:37687/ using document root <none> and password file <none>
I20260809 13:15:08.443631 14346 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:08.443733 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:08.444031 14346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:08.447081 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data/instance:
uuid: "87f8bb8c436f496daf303852400a21ce"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "2794a2798b564d46070383b633237d81"
server_key_iv: "b237810e5e3c873fe111917570edd5f6"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.447841 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal/instance:
uuid: "87f8bb8c436f496daf303852400a21ce"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "2794a2798b564d46070383b633237d81"
server_key_iv: "b237810e5e3c873fe111917570edd5f6"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.452668 14346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 13:15:08.456039 14368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:08.457283 14346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 13:15:08.457479 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal
uuid: "87f8bb8c436f496daf303852400a21ce"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "2794a2798b564d46070383b633237d81"
server_key_iv: "b237810e5e3c873fe111917570edd5f6"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.457685 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:08.472615 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:08.476321 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:08.476583 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:08.494095 14346 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.189:32953
I20260809 13:15:08.494130 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.189:32953 every 8 connection(s)
I20260809 13:15:08.495373 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/data/info.pb
I20260809 13:15:08.497135 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14346
I20260809 13:15:08.497334 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-1/wal/instance
I20260809 13:15:08.497916 10966 external_mini_cluster.cc:1428] Setting key 0dbe8853a17c676c2d29a99c190957ab
I20260809 13:15:08.499702 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:08.507942 14449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:08.529759 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.510908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.10.181.190:39001)
0809 13:15:08.511119 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:08.511125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:08.512282 (+ 1157us) server_negotiation.cc:407] Connection header received
0809 13:15:08.514268 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:08.514274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:08.514358 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:08.514447 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:08.516161 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.516865 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.518958 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.519389 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.519487 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.523430 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:08.523439 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:08.523444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:08.523492 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:08.526597 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.527308 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.527313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.527316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.527482 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.527893 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.527898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.527902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.528306 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:08.528455 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:08.529168 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 13:15:08.529421 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 13:15:08.530450 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.511354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39001 (local address 127.0.0.1:35050)
0809 13:15:08.512092 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.512148 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:08.514007 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:08.514691 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:08.514699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:08.515236 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:08.515977 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.516003 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.517041 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.517046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.518809 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.518826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.519620 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.519627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.519884 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.520551 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:08.520592 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:08.523147 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:08.526739 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.526747 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.526759 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.527175 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.527574 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.527578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.527581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.527766 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.528408 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:08.528413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:08.528759 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 13:15:08.529013 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 13:15:08.529375 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":308,"threads_started":1}
I20260809 13:15:08.532193 14449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.181.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.181.188@KRBTEST.COM" created.
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:08.546468 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.532806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51334)
0809 13:15:08.532968 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.532988 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:08.533085 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:08.534047 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:08.534051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:08.534442 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:08.534583 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.534592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.535974 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.535979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.536901 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.536917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.537964 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.537969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.538127 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.538634 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:08.538650 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:08.540607 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:08.544127 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.544131 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.544133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.544477 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.544892 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.544895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.544897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.545036 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.545766 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:08.545772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:08.545909 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 13:15:08.546022 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 13:15:08.546213 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:15:08.546837 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.533059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.10.181.189:32953)
0809 13:15:08.533659 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:08.533665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:08.533690 (+ 25us) server_negotiation.cc:407] Connection header received
0809 13:15:08.533730 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:08.533736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:08.533902 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:08.534027 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:08.534740 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.535847 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.537027 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.537819 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.537941 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.540819 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:08.540833 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:08.540849 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:08.540891 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:08.544010 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.544630 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.544638 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.544641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.544781 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.545137 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.545142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.545144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.545613 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:08.545795 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:08.546195 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 13:15:08.546442 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":289,"threads_started":1}
I20260809 13:15:08.549131 14449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:08.550361 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.549841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46122)
0809 13:15:08.550011 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.550189 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 13:15:08.550669 14374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.552230 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.551886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46128)
0809 13:15:08.552048 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.552133 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 13:15:08.552475 14449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
Entry for principal kudu/127.10.181.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab.
I20260809 13:15:08.574945 10966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
/tmp/dist-test-taskPW_N6O/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:35503
--webserver_interface=127.10.181.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.181.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 13:15:08.601785 14449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 1
I20260809 13:15:08.602893 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.602424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46144)
0809 13:15:08.602648 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.602736 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 13:15:08.603868 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.603531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46152)
0809 13:15:08.603700 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.603768 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 13:15:08.604066 14449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.672273 14449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 2
I20260809 13:15:08.673306 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.672858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46158)
0809 13:15:08.673056 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.673150 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 13:15:08.674279 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.673950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46168)
0809 13:15:08.674096 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.674164 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 13:15:08.674484 14449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.684818 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } attempt: 5
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:08.700762 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.685432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51350)
0809 13:15:08.685646 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.685665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:08.685805 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:08.686159 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:08.686164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:08.686544 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:08.686714 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.686723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.687705 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.687710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.688595 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:08.688608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.689335 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.689341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:08.689561 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.693591 (+ 4030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:08.693611 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:08.695586 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:08.698442 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.698447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.698452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.698794 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.699156 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:08.699161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:08.699164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:08.699305 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:08.699836 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:08.699840 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:08.699984 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 13:15:08.700104 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 13:15:08.700416 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 13:15:08.700762 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.685486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.10.181.189:32953)
0809 13:15:08.685655 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:08.685661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:08.685737 (+ 76us) server_negotiation.cc:407] Connection header received
0809 13:15:08.685922 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:08.685928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:08.686011 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:08.686096 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:08.686862 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.687567 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.688748 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:08.689189 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:08.689304 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:08.695764 (+ 6460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:08.695769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:08.695773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:08.695805 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:08.698300 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.698908 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.698912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.698915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.699041 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:08.699423 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:08.699427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:08.699429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:08.699718 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:08.699852 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:08.700216 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 13:15:08.700421 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 13:15:08.701958 14337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:08.703080 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.702622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46172)
0809 13:15:08.702813 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.702912 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 13:15:08.704033 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.703651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46178)
0809 13:15:08.703794 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.703880 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 13:15:08.704241 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.755458 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 1
I20260809 13:15:08.756675 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.756080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46180)
0809 13:15:08.756317 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.756450 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 13:15:08.757769 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.757340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46194)
0809 13:15:08.757527 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.757627 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 13:15:08.757987 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.765676 14449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 3
I20260809 13:15:08.766631 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.766224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46202)
0809 13:15:08.766396 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.766482 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 13:15:08.767596 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.767205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46218)
0809 13:15:08.767419 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.767492 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 13:15:08.767802 14449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
I20260809 13:15:08.826220 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 2
I20260809 13:15:08.827536 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.826946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46226)
0809 13:15:08.827197 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.827325 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 13:15:08.828586 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.828193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46230)
0809 13:15:08.828350 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.828432 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 13:15:08.828814 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
W20260809 13:15:08.910382 14461 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 13:15:08.910678 14461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 13:15:08.910781 14461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 13:15:08.910893 14461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 13:15:08.911022 14449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 4
I20260809 13:15:08.912166 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.911692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46240)
0809 13:15:08.911911 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.912002 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 13:15:08.913074 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.912791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46246)
0809 13:15:08.912905 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.912972 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 13:15:08.913261 14449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
W20260809 13:15:08.918617 14461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 13:15:08.918692 14461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 13:15:08.918778 14461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 13:15:08.918835 14461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 13:15:08.918916 14461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 13:15:08.928462 14461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.181.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.181.190:39001,127.10.181.189:32953,127.10.181.188:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/krb5kdc/kudu_127.10.181.188.keytab
--principal=kudu/127.10.181.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.181.188:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.181.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:32:11 UTC on ccycloud.quasar-ficarf.root.comops.site
build id 4575995
ASAN enabled
I20260809 13:15:08.929670 14461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 13:15:08.930950 14461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 13:15:08.935041 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 3
I20260809 13:15:08.936193 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.935666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46256)
0809 13:15:08.935880 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.935988 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 13:15:08.937368 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:08.936928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46272)
0809 13:15:08.937139 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:08.937230 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 13:15:08.937641 14337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.181.188:35503: Network error: Client connection negotiation failed: client connection to 127.10.181.188:35503: connect: Connection refused (error 111)
W20260809 13:15:08.943279 14469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:08.943305 14476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 13:15:08.943240 14471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 13:15:08.944283 14461 server_base.cc:1034] running on GCE node
I20260809 13:15:08.945024 14461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 13:15:08.946391 14461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 13:15:08.947613 14461 hybrid_clock.cc:648] HybridClock initialized: now 1786281308947544 us; error 34 us; skew 500 ppm
I20260809 13:15:08.947928 14461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 13:15:08 dist-test-slave-42z9 krb5kdc[14209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 13:15:08.950914 14461 init.cc:377] Logged in from keytab as kudu/127.10.181.188@KRBTEST.COM (short username kudu)
I20260809 13:15:08.952250 14461 webserver.cc:460] Webserver started at http://127.10.181.188:41617/ using document root <none> and password file <none>
I20260809 13:15:08.952945 14461 fs_manager.cc:359] Metadata directory not provided
I20260809 13:15:08.953044 14461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 13:15:08.953347 14461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 13:15:08.956231 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data/instance:
uuid: "36a457a3d00d4bcfb3c4e0017dec8697"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "ecf350bc9d26932cea16a8c91e22c6e5"
server_key_iv: "1f74123ddf3768e3ee0739bd8aa971ea"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.956987 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal/instance:
uuid: "36a457a3d00d4bcfb3c4e0017dec8697"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "ecf350bc9d26932cea16a8c91e22c6e5"
server_key_iv: "1f74123ddf3768e3ee0739bd8aa971ea"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.962028 14461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 13:15:08.965305 14485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 13:15:08.966531 14461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 13:15:08.966704 14461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data,/tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal
uuid: "36a457a3d00d4bcfb3c4e0017dec8697"
format_stamp: "Formatted at 2026-08-09 13:15:08 on dist-test-slave-42z9"
server_key: "ecf350bc9d26932cea16a8c91e22c6e5"
server_key_iv: "1f74123ddf3768e3ee0739bd8aa971ea"
server_key_version: "encryptionkey@0"
I20260809 13:15:08.966895 14461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 13:15:08.997130 14461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 13:15:09.001274 14461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 13:15:09.001555 14461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 13:15:09.018253 14461 rpc_server.cc:307] RPC server started. Bound to: 127.10.181.188:35503
I20260809 13:15:09.018275 14565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.181.188:35503 every 8 connection(s)
I20260809 13:15:09.019570 14461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/data/info.pb
I20260809 13:15:09.023195 14566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 13:15:09.025504 10966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu as pid 14461
I20260809 13:15:09.025688 10966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPW_N6O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786280946828509-10966-0/minicluster-data/master-2/wal/instance
I20260809 13:15:09.026214 10966 external_mini_cluster.cc:1428] Setting key c6d97a96b70cb906c03c82e33408eccf
I20260809 13:15:09.031055 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:09.049109 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.033347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.10.181.190:39001)
0809 13:15:09.033847 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.033853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.034366 (+ 513us) server_negotiation.cc:407] Connection header received
0809 13:15:09.034523 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.034527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.034598 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.034688 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.035901 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.036872 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.038602 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.039232 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.039367 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.042565 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.042573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.042577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.042620 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.045915 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.046708 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.046712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.046715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.046866 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.047205 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.047210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.047213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.047550 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.047691 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.048482 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.048710 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260809 13:15:09.049566 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.028067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.10.181.188:35503)
0809 13:15:09.028837 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.028849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.028894 (+ 45us) server_negotiation.cc:407] Connection header received
0809 13:15:09.030811 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.030836 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.031222 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.031607 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.033055 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.035072 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.036583 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.037414 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.037615 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.041640 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.041648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.041661 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.041706 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.045250 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.046003 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.046012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.046021 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.046208 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.046680 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.046685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.046688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.047345 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.047652 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.047993 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.048397 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":309,"threads_started":1}
I20260809 13:15:09.049599 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.033383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.190:39001 (local address 127.0.0.1:35066)
0809 13:15:09.034216 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:09.034272 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:09.034417 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:09.034757 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:09.034764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:09.035494 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:09.035745 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.035754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.037031 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.037036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.038459 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.038473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.039442 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.039447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.039567 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.040134 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:09.040158 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:09.042379 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:09.046037 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.046050 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.046058 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.046584 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.046966 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.046969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.046971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.047097 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.047685 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:09.047693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:09.047934 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 13:15:09.048150 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 13:15:09.048526 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":75,"thread_start_us":453,"threads_started":1}
I20260809 13:15:09.051100 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.190@KRBTEST.COM
I20260809 13:15:09.064652 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.051687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.189:32953 (local address 127.0.0.1:51352)
0809 13:15:09.051823 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:09.051839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:09.051911 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:09.052311 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:09.052316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:09.052688 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:09.052865 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.052877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.054194 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.054198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.055456 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.055470 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.056393 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.056397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.056555 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.057016 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:09.057032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:09.059233 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:09.062527 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.062531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.062533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.062885 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.063281 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.063285 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.063287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.063420 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.064000 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:09.064003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:09.064120 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 13:15:09.064234 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 13:15:09.064426 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 13:15:09.065021 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.051850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.10.181.189:32953)
0809 13:15:09.052028 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.052033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.052052 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:15:09.052108 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.052114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.052177 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.052252 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.053000 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.054061 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.055603 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.056267 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.056395 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.059431 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.059436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.059440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.059473 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.062392 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.063001 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.063007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.063011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.063163 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.063523 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.063528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.063531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.063883 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.064049 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.064361 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.064567 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 13:15:09.065863 14566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } has no permanent_uuid. Determining permanent_uuid...
I20260809 13:15:09.067466 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.052654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.10.181.190:39001)
0809 13:15:09.052812 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.052818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.052837 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:15:09.052930 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.052935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.053017 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.053093 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.054449 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.055454 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.056957 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.057364 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.057491 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.061603 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.061610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.061615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.061662 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.064567 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.065397 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.065401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.065403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.065557 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.065976 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.065981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.065983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.066341 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.066607 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.066954 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.067172 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.188@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.181.189@KRBTEST.COM
I20260809 13:15:09.078469 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.066598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46294)
0809 13:15:09.066880 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:09.066897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:09.066990 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:09.067311 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:09.067315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:09.068057 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:09.068243 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.068255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.069208 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.069214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.070309 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.070322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.071153 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.071159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.071374 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.071870 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:09.071891 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:09.073716 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:09.076315 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.076320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.076324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.076648 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.076967 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.076972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.076975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.077159 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.077698 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:09.077703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:09.077844 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 13:15:09.077955 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 13:15:09.078175 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 13:15:09.078670 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.066715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.10.181.188:35503)
0809 13:15:09.066864 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.066869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.066939 (+ 70us) server_negotiation.cc:407] Connection header received
0809 13:15:09.067090 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.067095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.067173 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.067430 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.068402 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.069077 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.070442 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.070844 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.070981 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.073891 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.073896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.073899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.073927 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.076199 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.076749 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.076752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.076755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.076867 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.077263 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.077267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.077269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.077568 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.077697 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.078074 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.078342 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":48}
I20260809 13:15:09.083129 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.069279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.10.181.189:32953)
0809 13:15:09.069486 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.069491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.069508 (+ 17us) server_negotiation.cc:407] Connection header received
0809 13:15:09.069553 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.069558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.069623 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.069748 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.070868 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.071855 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.073009 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.073511 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.073621 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.077767 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.077774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.077778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.077808 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.080717 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.081417 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.081420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.081423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.081607 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.082005 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.082008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.082010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.082242 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.082420 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.082734 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.082887 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 13:15:09.084136 14566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: Bootstrap starting.
I20260809 13:15:09.087754 10966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 13:15:09.088160 14566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:09.089412 14566 log.cc:824] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:09.092244 14566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: No bootstrap required, opened a new log
I20260809 13:15:09.090790 14296 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "74ce6d8219f645acb66cf5c529005721"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35078
I20260809 13:15:09.099185 14566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.099519 14566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a457a3d00d4bcfb3c4e0017dec8697, State: Initialized, Role: FOLLOWER
I20260809 13:15:09.099964 14566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.100786 14577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.101084 14577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.102065 14566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 13:15:09.111035 14591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:15:09.111172 14591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:15:09.114835 14337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 4
I20260809 13:15:09.108697 14461 master_runner.cc:186] Error getting master registration for 127.10.181.190:39001: OK, instance_id { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" instance_seqno: 1786281307980177 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:15:09.119629 14461 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281307, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.190@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
I20260809 13:15:09.131009 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.115369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46306)
0809 13:15:09.115660 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:09.115680 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:09.115788 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:09.116326 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:09.116331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:09.116704 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:09.116879 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.116888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.118345 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.118350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.119264 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.119276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.120325 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.120330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.120524 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.121143 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:09.121164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:09.123992 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:09.128171 (+ 4179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.128177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.128181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.128687 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.129164 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.129170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.129174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.129349 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.130049 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:09.130055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:09.130212 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 13:15:09.130337 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 13:15:09.130622 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260809 13:15:09.131019 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.115456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.10.181.188:35503)
0809 13:15:09.116025 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.116034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.116052 (+ 18us) server_negotiation.cc:407] Connection header received
0809 13:15:09.116108 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.116113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.116189 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.116285 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.117031 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.118205 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.119418 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.120171 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.120302 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.124184 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.124191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.124196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.124239 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.128049 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.128813 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.128840 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.128844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.129024 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.129488 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.129493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.129497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.129913 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.130070 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.130464 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.130677 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421}
I20260809 13:15:09.135593 14337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: Bootstrap starting.
I20260809 13:15:09.139503 14337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:09.140771 14337 log.cc:824] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:09.143420 14337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: No bootstrap required, opened a new log
I20260809 13:15:09.150516 14337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.150918 14337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ce6d8219f645acb66cf5c529005721, State: Initialized, Role: FOLLOWER
I20260809 13:15:09.151414 14337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.152652 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.153257 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.153750 14337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:15:09.154304 14296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 13:15:09.157123 14296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:09.160472 14296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.163832 14524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ce6d8219f645acb66cf5c529005721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "36a457a3d00d4bcfb3c4e0017dec8697"
I20260809 13:15:09.164371 14524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:09.166967 14524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74ce6d8219f645acb66cf5c529005721 in term 1.
I20260809 13:15:09.168378 14257 leader_election.cc:302] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36a457a3d00d4bcfb3c4e0017dec8697, 74ce6d8219f645acb66cf5c529005721; no voters:
I20260809 13:15:09.169196 14598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 13:15:09.170173 14598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [term 1 LEADER]: Becoming Leader. State: Replica: 74ce6d8219f645acb66cf5c529005721, State: Running, Role: LEADER
W20260809 13:15:09.171053 14611 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 13:15:09.171307 14611 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 13:15:09.171099 14598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.174343 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ce6d8219f645acb66cf5c529005721. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.174649 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:09.171136 14296 leader_election.cc:288] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [CANDIDATE]: Term 1 election: Requested vote from peers 87f8bb8c436f496daf303852400a21ce (127.10.181.189:32953), 36a457a3d00d4bcfb3c4e0017dec8697 (127.10.181.188:35503)
I20260809 13:15:09.184983 14619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 13:15:09.187510 14619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 13:15:09.177573 14229 master_runner.cc:186] Error getting master registration for 127.10.181.189:32953: OK, instance_id { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" instance_seqno: 1786281308489219 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 13:15:09.197211 14229 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 13:15:09.204135 14524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [term 1 FOLLOWER]: Refusing update from remote peer 74ce6d8219f645acb66cf5c529005721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:15:09.205636 14598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:15:09.212441 14449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } attempt: 5
I20260809 13:15:09.219426 14577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ce6d8219f645acb66cf5c529005721. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.219801 14577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 13:15:09.220535 14260 tablet.cc:2369] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 13:15:09 dist-test-slave-42z9 krb5kdc[14209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786281308, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.181.189@KRBTEST.COM for kudu/127.10.181.188@KRBTEST.COM
I20260809 13:15:09.229311 14622 mvcc.cc:205] Tried to move back new op lower bound from 7316608242486689792 to 7316608242370105344. Current Snapshot: MvccSnapshot[applied={T|T < 7316608242486689792}]
I20260809 13:15:09.229880 14623 mvcc.cc:205] Tried to move back new op lower bound from 7316608242486689792 to 7316608242370105344. Current Snapshot: MvccSnapshot[applied={T|T < 7316608242486689792}]
I20260809 13:15:09.232429 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.213018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.181.188:35503 (local address 127.0.0.1:46316)
0809 13:15:09.213373 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 13:15:09.213393 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 13:15:09.213530 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 13:15:09.214603 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 13:15:09.214609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 13:15:09.215143 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 13:15:09.215359 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.215372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.216858 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.216865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.218259 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 13:15:09.218277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.221060 (+ 2783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.221066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 13:15:09.221286 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.221969 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 13:15:09.221994 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 13:15:09.224864 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 13:15:09.229059 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.229064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.229068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.229925 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.230412 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 13:15:09.230417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 13:15:09.230422 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 13:15:09.230657 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 13:15:09.231382 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 13:15:09.231388 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 13:15:09.231560 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 13:15:09.231713 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 13:15:09.232019 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260809 13:15:09.233387 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.213891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.10.181.188:35503)
0809 13:15:09.214089 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.214095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.214116 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:15:09.214174 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.214179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.214259 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.214363 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.215543 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.216715 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.218409 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.220661 (+ 2252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.220795 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.225082 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.225089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.225094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.225139 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.228896 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.230075 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.230081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.230085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.230266 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.230800 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.230806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.230809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.231173 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.231315 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.232800 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.233046 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 13:15:09.239255 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: Bootstrap starting.
I20260809 13:15:09.242342 14577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.242617 14577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.243635 14449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: Neither blocks nor log segments found. Creating new log.
I20260809 13:15:09.244982 14449 log.cc:824] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: Log is configured to *not* fsync() on all Append() calls
I20260809 13:15:09.248642 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.248914 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: This master's current role is: LEADER
I20260809 13:15:09.249686 14619 catalog_manager.cc:1347] Generated new cluster ID: 184b2b0189d5451196574e8fb469a66e
I20260809 13:15:09.249780 14619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 13:15:09.294951 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: No bootstrap required, opened a new log
I20260809 13:15:09.302661 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.270597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.10.181.189:32953)
0809 13:15:09.270767 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.270772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.279317 (+ 8545us) server_negotiation.cc:407] Connection header received
0809 13:15:09.279651 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.279658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.279740 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.279811 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.282702 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.283801 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.286033 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.286695 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.286793 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.295274 (+ 8481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.295282 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.295287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.295328 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.299205 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.300074 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.300080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.300084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.300254 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.300707 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.300713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.300716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.301107 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.301365 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.301977 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.302294 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 13:15:09.309422 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.262369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35090 (local address 127.10.181.190:39001)
0809 13:15:09.262569 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.262574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.279262 (+ 16688us) server_negotiation.cc:407] Connection header received
0809 13:15:09.279552 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.279558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.279660 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.279735 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.283195 (+ 3460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.284332 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.286243 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.286906 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.287006 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.301787 (+ 14781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.301795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.301800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.301843 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.305430 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.306291 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.306297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.306301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.306489 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.307461 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.307467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.307471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.307874 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.308207 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.308769 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.309012 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 13:15:09.323506 14619 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 13:15:09.325465 14619 catalog_manager.cc:1504] Loading token signing keys...
I20260809 13:15:09.327430 14449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.327883 14449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8bb8c436f496daf303852400a21ce, State: Initialized, Role: FOLLOWER
I20260809 13:15:09.328444 14449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } }
I20260809 13:15:09.331704 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:09.247092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.10.181.188:35503)
0809 13:15:09.247274 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.247279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.270006 (+ 22727us) server_negotiation.cc:407] Connection header received
0809 13:15:09.277960 (+ 7954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.277966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.278067 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.278186 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.309779 (+ 31593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.310903 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.313339 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:09.314051 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:09.314160 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:09.317419 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 13:15:09.317426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 13:15:09.317431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 13:15:09.317491 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 13:15:09.321269 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.329056 (+ 7787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.329062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.329066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.329243 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 13:15:09.329811 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 13:15:09.329816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 13:15:09.329820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 13:15:09.330175 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 13:15:09.330449 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:09.331038 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 13:15:09.331309 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 13:15:09.333035 14644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.333478 14644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.335391 14449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 13:15:09.335788 14406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 13:15:09.336256 14407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ce6d8219f645acb66cf5c529005721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "87f8bb8c436f496daf303852400a21ce"
I20260809 13:15:09.340502 14406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [term 1 FOLLOWER]: Refusing update from remote peer 74ce6d8219f645acb66cf5c529005721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 13:15:09.344676 14619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721: Generated new TSK 0
I20260809 13:15:09.344898 14407 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74ce6d8219f645acb66cf5c529005721 in term 1.
I20260809 13:15:09.347890 14598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 13:15:09.348532 14619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 13:15:09.378158 14648 mvcc.cc:205] Tried to move back new op lower bound from 7316608242486689792 to 7316608242370105344. Current Snapshot: MvccSnapshot[applied={T|T < 7316608242486689792}]
I20260809 13:15:09.392903 14644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ce6d8219f645acb66cf5c529005721. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.393285 14644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.392886 14655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.393553 14655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 13:15:09.420140 14672 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 13:15:09.448910 14573 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.442664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.10.181.190:39001)
0809 13:15:09.442855 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.442860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.447058 (+ 4198us) server_negotiation.cc:407] Connection header received
0809 13:15:09.447344 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.447350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.447436 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.447507 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.448657 (+ 1150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35100: BlockingRecv error: recv got EOF from 127.0.0.1:35100 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 13:15:09.450222 14570 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.443132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.10.181.188:35503)
0809 13:15:09.443311 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.443316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.446766 (+ 3450us) server_negotiation.cc:407] Connection header received
0809 13:15:09.448103 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.448113 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.448196 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.448262 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.449979 (+ 1717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46328: BlockingRecv error: recv got EOF from 127.0.0.1:46328 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 13:15:09.450980 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.447258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.10.181.189:32953)
0809 13:15:09.447434 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.447440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.449434 (+ 1994us) server_negotiation.cc:407] Connection header received
0809 13:15:09.449733 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.449738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.449827 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.449892 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.450766 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51364: BlockingRecv error: recv got EOF from 127.0.0.1:51364 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 13:15:09.457897 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.455606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51380 (local address 127.10.181.189:32953)
0809 13:15:09.455771 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.455776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.456539 (+ 763us) server_negotiation.cc:407] Connection header received
0809 13:15:09.456739 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.456745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.456865 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.456968 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.457669 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51380: BlockingRecv error: recv got EOF from 127.0.0.1:51380 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 13:15:09.465399 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.463634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.10.181.189:32953)
0809 13:15:09.463770 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.463776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.464209 (+ 433us) server_negotiation.cc:407] Connection header received
0809 13:15:09.464339 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.464345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.464424 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.464522 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.465198 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51392: BlockingRecv error: recv got EOF from 127.0.0.1:51392 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 13:15:09.473366 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.471429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.10.181.189:32953)
0809 13:15:09.471592 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.471597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.472107 (+ 510us) server_negotiation.cc:407] Connection header received
0809 13:15:09.472249 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.472255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.472354 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.472456 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.473154 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51404: BlockingRecv error: recv got EOF from 127.0.0.1:51404 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 13:15:09.488973 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.486915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.10.181.189:32953)
0809 13:15:09.487111 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.487117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.487648 (+ 531us) server_negotiation.cc:407] Connection header received
0809 13:15:09.487833 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.487838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.487921 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.488018 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.488778 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51408: BlockingRecv error: recv got EOF from 127.0.0.1:51408 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260809 13:15:09.525417 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.523058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.10.181.189:32953)
0809 13:15:09.523264 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.523271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.523919 (+ 648us) server_negotiation.cc:407] Connection header received
0809 13:15:09.524122 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.524129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.524226 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.524337 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.525172 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51420: BlockingRecv error: recv got EOF from 127.0.0.1:51420 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 13:15:09.547578 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.544883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.10.181.189:32953)
0809 13:15:09.545086 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.545092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.545886 (+ 794us) server_negotiation.cc:407] Connection header received
0809 13:15:09.546187 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.546193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.546306 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.546422 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.547338 (+ 916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51424: BlockingRecv error: recv got EOF from 127.0.0.1:51424 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 13:15:09.585492 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.583042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.10.181.189:32953)
0809 13:15:09.583232 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.583238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.583831 (+ 593us) server_negotiation.cc:407] Connection header received
0809 13:15:09.584047 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.584054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.584162 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.584284 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.585189 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51432: BlockingRecv error: recv got EOF from 127.0.0.1:51432 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 13:15:09.654493 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.652552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.10.181.189:32953)
0809 13:15:09.652767 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.652773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.653091 (+ 318us) server_negotiation.cc:407] Connection header received
0809 13:15:09.653266 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.653272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.653369 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.653515 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.654250 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51968: BlockingRecv error: recv got EOF from 127.0.0.1:51968 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 13:15:09.787569 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:09.785605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.10.181.189:32953)
0809 13:15:09.785816 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:09.785823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:09.786200 (+ 377us) server_negotiation.cc:407] Connection header received
0809 13:15:09.786405 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:09.786409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:09.786488 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:09.786578 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:09.787377 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51978: BlockingRecv error: recv got EOF from 127.0.0.1:51978 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 13:15:09.824724 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74ce6d8219f645acb66cf5c529005721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ce6d8219f645acb66cf5c529005721" member_type: VOTER last_known_addr { host: "127.10.181.190" port: 39001 } } peers { permanent_uuid: "87f8bb8c436f496daf303852400a21ce" member_type: VOTER last_known_addr { host: "127.10.181.189" port: 32953 } } peers { permanent_uuid: "36a457a3d00d4bcfb3c4e0017dec8697" member_type: VOTER last_known_addr { host: "127.10.181.188" port: 35503 } } }
I20260809 13:15:09.825083 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ce6d8219f645acb66cf5c529005721 [sys.catalog]: This master's current role is: LEADER
W20260809 13:15:10.048674 14455 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 13:15:10.047037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.10.181.189:32953)
0809 13:15:10.047221 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.047225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.047494 (+ 269us) server_negotiation.cc:407] Connection header received
0809 13:15:10.047662 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.047666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.047759 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.047857 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 13:15:10.048495 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51980: BlockingRecv error: recv got EOF from 127.0.0.1:51980 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 13:15:10.113242 14591 catalog_manager.cc:1259] Loaded cluster ID: 184b2b0189d5451196574e8fb469a66e
I20260809 13:15:10.113349 14591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: loading cluster ID for follower catalog manager: success
I20260809 13:15:10.115896 14591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: acquiring CA information for follower catalog manager: success
I20260809 13:15:10.117874 14591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36a457a3d00d4bcfb3c4e0017dec8697: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:15:10.424638 14672 catalog_manager.cc:1259] Loaded cluster ID: 184b2b0189d5451196574e8fb469a66e
I20260809 13:15:10.424751 14672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: loading cluster ID for follower catalog manager: success
I20260809 13:15:10.427371 14672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: acquiring CA information for follower catalog manager: success
I20260809 13:15:10.429477 14672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87f8bb8c436f496daf303852400a21ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 13:15:10.572798 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:10.564547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.10.181.189:32953)
0809 13:15:10.565944 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.565950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.566498 (+ 548us) server_negotiation.cc:407] Connection header received
0809 13:15:10.566721 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.566724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.566895 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.567012 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:10.567537 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.568375 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.570374 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.570933 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.571026 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:10.571606 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:10.572178 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:10.572289 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:10.572419 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 13:15:10.572492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":1071,"threads_started":1}
I20260809 13:15:10.584740 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:10.579200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.10.181.188:35503)
0809 13:15:10.579830 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.579836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.579855 (+ 19us) server_negotiation.cc:407] Connection header received
0809 13:15:10.579958 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.579962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.580127 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.580279 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:10.580683 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.581408 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.582689 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.583078 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.583199 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:10.583656 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:10.584127 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:10.584263 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:10.584404 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 13:15:10.584488 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":385,"threads_started":1}
I20260809 13:15:10.612534 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:10.597976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.10.181.188:35503)
0809 13:15:10.598174 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.598180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.606342 (+ 8162us) server_negotiation.cc:407] Connection header received
0809 13:15:10.606602 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.606608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.606712 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.606776 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:10.607506 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.608489 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.610561 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.611155 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.611242 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:10.611827 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:10.611940 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:10.611993 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:10.612187 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 13:15:10.612262 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 13:15:10.613039 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:10.597817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.10.181.190:39001)
0809 13:15:10.605094 (+ 7277us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.605100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.605121 (+ 21us) server_negotiation.cc:407] Connection header received
0809 13:15:10.605228 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.605234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.605430 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.605651 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:10.606324 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.607446 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.609805 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.610461 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.610559 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:10.611350 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:10.611829 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:10.611932 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:10.612608 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 13:15:10.612718 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7154,"thread_start_us":7049,"threads_started":1}
I20260809 13:15:10.617586 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14229
I20260809 13:15:10.618562 14329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:10.619035 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 13:15:10.606415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.10.181.189:32953)
0809 13:15:10.606576 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 13:15:10.606582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 13:15:10.612515 (+ 5933us) server_negotiation.cc:407] Connection header received
0809 13:15:10.613547 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 13:15:10.613553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 13:15:10.613655 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 13:15:10.613720 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 13:15:10.614281 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.615321 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.617000 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 13:15:10.617658 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 13:15:10.617776 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 13:15:10.618309 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 13:15:10.618418 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 13:15:10.618505 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 13:15:10.618667 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 13:15:10.618750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 13:15:10.729857 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14229
I20260809 13:15:10.770756 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14346
I20260809 13:15:10.772002 14444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:10.883322 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14346
I20260809 13:15:10.910072 10966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu pid 14461
I20260809 13:15:10.911191 14557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 13:15:11.016662 10966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPW_N6O/build/asan/bin/kudu with pid 14461
2026-08-09T13:15:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5010 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10064 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364158 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST